Sherpa

=============

This release fixes several bugs and introduces a few new features, notably the ability to evaluate model components on arbitrary grids and generate user-defined ARFs and RMFs. Also, as of this release Sherpa will no longer rely on any Fortran code. See the following section for details.

It is now possible to build the Sherpa documentation using Sphinx. Additionally, the Sphinx documentation is
automatically built and hosted on ReadTheDocs: https://sherpa.readthedocs.io/

Details
-------

407 Ensure 0-length array is an error in filter_resp (fix 405)
Add an explicit check in the C++ filter_resp code to error out if the noticed channels array
is empty.

422 Improve error message with wrong xspec version
Improve the handling of XSPEC versions mismatch.

466 Fix bounding box out-of-bounds memory read
Avoid an out-of-bounds memory read when calling pad_bounding_box (when the data is not
matching the expected conditions for this call).

469 Evaluate model on finer grid
Sherpa users can now define arbitrary grids, called evaluation spaces, on which to
evaluate individual model components, both in 1D and 2D. This can be useful in a number
of cases, for instance when it is desirable to evaluate models on a finer grid than the one
defined by the data, or in convolution models where information outside of the data range
can be used to reduce boundary effects or to inform the evaluation inside the data space.
Also, when plotting individual 1D model components (plot_source_component), if a specific
evaluation space was attached to the model then the plot will be of the model evaluated
over that evaluation space, not the data space. Other plotting commands should be
unaffected by this change.

470 Order for np.ones() should be a one length string not a bool
Fixed a `DeprecationWarning` in the optimization module.

471 Using overwrite rather than clobber of astropy.io.fits
`clobber` in astropy has been deprecated for a while now in favour of overwrite and thus
issues a `DeprecationWarning`. Sherpa now uses `overwrite` instead.

475 Fix unecessary runtime warning (fix 402)
A condition check in the optimization code was modified so as not to produce a warning
when the value is a NaN.

481 Remove Fortran code
Sherpa does not rely on any Fortran routines anymore, except those compiled in XSPEC.
The existing Fortran code, which was used mainly in some of the optimization routines, has
been replaced by equivalent C++ code. This means that gfortran is no longer required for
building Sherpa, although a version of libgfortran compatible with the one used to link the
XSPEC libraries would still be needed at runtime if the XSPEC extension is
built.

482 ARF/RMF creation functions
The `create_rmf` and `create_arf` functions now allow users to easily generate user
defined response objects.

486 `ZeroDivisionError` in `calc_stat_info` (fix 476)
Sherpa did not catch divisions by zero in the calc_stat_info command. That has been fixed.

484 Equivalent Width errors
Several optional arguments `(params=None, error=False, otherids=(), niter=1000)`, were
added to `eqwidth`. If `error = True`, then get_draws shall be run if the fit stat is one of the
following: (Cash, CStat, WStat) otherwise a multivariate normal distribution shall be run to
generate the samples. The optional niter parameter is used to generate the number of
samples. The optional parameter otherids is only used if get_draws is used internally when
multiple data sets were used to fit. Alternatively, the user can enter the samples via the
`params` option where the samples must be a `numpy.ndarray` of dimension `(num, npar)`.

487 PSF bin size warning
Sherpa assumes that the PSF image and the data have the same pixel size. When this is
not true Sherpa ignores the difference, which results in a larger PSF being applied. From
now on Sherpa will issue a warning when the PSF image
pixel size and the data image pixel size are different.

Testing code now works with pip 10 as well, despite a change in the pip 10 API. Also, the README is now properly rendered as markdown in PyPI.

438 Change from error to warning for OGIP violations
Given that users can not easily change a response file, and previous versions of Sherpa would allow the responses to be used, this commit changes some of the errors recently introduced (in PR 383) into warnings. The errors for the first bin edge being <= 0 are still left in because users of the sherpa.astro.ui module will find that these files are auto-corrected for them (by PR 383).

430 Update XSPEC parameters
XSPEC model parameter default values, limits, and properties were reviewed and updated to reflect changes in or mismatches with the model.dat file shipped with XSPEC 12.9.1n.

428 handle function name changes in XSPEC 12.9.1
Sherpa now supports multiple versions of the XSPEC models from the 12.9.0 and 12.9.1 series. Some models recommend using the C-style interface over the older FORTRAN one, which may also resolve some memory access issues. For CIAO 4.10 users this means the interfaces to XSPEC models have been updated to the 12.9.1n versions. For Standalone Sherpa users this means they can build and run Sherpa against a larger range of XSPEC versions, and Sherpa will pick the XSPEC low level model function accordingly. Note that Sherpa has been tested against XSPEC 12.9.0i, 12.9.0o, and 12.9.1n.
The 14 models that have been changed are: apec, bapec, bvapec, bvvapec, gaussian, lorentz, meka, mekal, raymond, vapec, vmeka, vmekal, vraymond, and vvapec.

427 Add support for XSPEC models added in 12.9.1 (fix 331)
Add support for models added in XSPEC 12.9.1: bvtapec, bvvtapec, tapec, vtapec, vvtapec, carbatm, hatm, ismabs, slimbh, snapec, TBfeo, TBgas, TBpcf, TBrel, voigt, xscat.
Version 12.9.0 of XSPEC can still be used, in which case the models can be generated - e.g. a user can say xstapec.mdl to create a Python object - but it will error out when evaluated.
The Si and S elemental parameters for the ismabs model have been renamed from the XSPEC versions since they are not unique using the caseinsensitive matching used by Sherpa: so SI, SII, SIII and SiI, SiII, SiIII have been renamed S_I, S_II, S_III and Si_I, Si_II, and Si_III respectively.
Low level support for the following convolution models from XSPEC 12.9.1 has also been added, but there is no Python model support for these: clumin, rfxconv, vashift, vmshift, xilconv.

412 support ROSAT PSPC (and similar) RMF files with AstroPy
Add explicit tests for reading in, and using, ROSAT PSPC PHA and RMF files. Fix a bug in the AstroPy back-end handling of the ROSAT RMF file.

409 EmissionGaussian model when skew parameter is not unity (fix 403)
The `sherpa.astro.optical.EmissionGaussian` code has been fixed for the case when the skew parameter is not unity. The documentation has also been updated.

399 Python 3 fixes and new tests for pha2 datasets
Fixed several problems when using Sherpa with Python 3: failures when calling `list_bkg_ids` and `list_response_ids` and the parameter querying after `paramprompt(True)` has been called.
There is also a change to avoid a `FutureWarning` being raised in the astropy backend when reading in PHA2 data files.
Tests have been added to test the reading of a PHA2 format dataset, and new files added to the `sherpa-test-data` repository for this purpose.

398 Check that list_samplers returns a list (fix 397)
The `list_samplers` function now returns a list in Python 3 as well.

396 Fix `set_xlog` and related commands using Python 3 (fix 393)
Fix use of commands that set the plot state using the `sherpa.astro.ui` module, such as `set_xlog` and `set_xlinear`, when using Python 3.

395 Add xspec optional models
Sherpa has new infrastructure for supporting multiple versions of XSPEC at the same time, with models that are built and enabled conditionally depending on the version of XSPEC being linked. Models are assumed to have the same parameters and low level functions across supported versions.

394 XSPEC build and functionality improvements
Add support for reading and changing the path to the XSPEC "manager" directory, and reading the current XSPEC "model" directory (`get_xspath_manager`, `set_xspath_manager`, and `get_xspath_model`). These are intended for the power user and so require an explicit import from `sherpa.astro.xspec`.
There are several improvements to the build and interface to the XSPEC model library: these have no user-visible changes.

390 Add nlapec model
The `nlapec` XSpec model has been added. Note that XSPEC 12.9.0 is now required.

385 DS9 calls return string in Python 2 and 3 (fix 319)
Some low lever `xpa` calls were returning byte strings rather than strings in Python 3. This had particular impact on the `image_getregion` function. This has now been fixed.

383 Replace 0-energy bins in ARF and RMFs (fix 332)
Sherpa now performs some validation of the energy ranges of ARF and RMF files against the OGIP standard. If the ranges are inconsistent then the code will return with an error. If one energy bound is 0 the bound is replaced with a small number to avoid numerical issues like division by zero. A new configuration option `minimum_energy`, if present, allows users to override Sherpa's default behavior.

379 Use a line not span to draw horizontal line (fix 378)
Ensure that the line at y=0 (residual) or y=1 (ratio) is drawn for residual or ratio plots when using matplotlib 2.0.

373 Make sure ds9 properly works with Python 2.7 (fix 368)
For Python versions prior to 3.2 add in explicit code to make _Popen work as a context manager.

352 Add low-level support for the XSPEC rgsxsrc convolution model
Add low-level support for the rgsxsrc convolution model in XSPEC. This is intended for R&D into fully supporting XSPEC convolution models in Sherpa.

255 Add aliases for parameter names so to support new xspec names (fix 74)
The XSPEC models have been updated to use parameter names that match the new XSPEC naming scheme introduced in XSPEC 12.9.0. The old parameter names can still be used as aliases for the new name, but are deprecated and may be removed in a next major release. Note this allows any models to define aliases for their parameters.

============

This version introduces full support for Python 3.6. It also fixes issues with non-Chandra response files, correctly
handles the AREASCAL column in PHA files, and fixes a significant regression that was preventing user statistics from
working in v4.8.2. It also introduces a number of smaller improvements and fixes. In particular, quite a few
improvements have been made to the documentation and to the testing framework, including several new tests to improve
stability.

Details
-------

Infrastructure and minor non-functional changes have been omitted.

335 Fix `setup.py` install command
The `setup.py install` command was not enforcing the installation of the dependencies listed in `setup.py`. This has
been fixed.

368 Remove ds9 warnings when run under Python 3.6
Update the DS9 code so that external processes are cleaned up properly, so to remove the potential ResourceWarning
warnings when running DS9 on Python 3.6.

351 fix handling of AREASCAL column in PHA files (fix 350)
Add support for handling the AREASCAL value (either scalar of vector) for PHA data sets. This array is used in XMM
RGS data to handle missing chips.

358 Properly handle Swift RMF when using Astropy backend (fix 357)
A Swift RMF could not be read in when the AstroPy back end was in use. The problem was that the code did not support
RMF matrices that were not stored as variable-length arrays. This has now been fixed, and new tests have been added
for this kind of files.

343 Fix user statistics regression (fix 341)
A number of regressions were introduced in version 4.8.1 up to version 4.9.0, so user statistics that were properly
working in version 4.7 have not been working any more. This has been fixed, and a number of regression tests have
been added.

This version fixes many bugs in the Python 3 support. Moreover, it includes a significant
refactoring of the `Fit` and `Stat` classes that made it possible to fix several bugs
related to the recent `wstat` implementation while making these classes more maintainable
and extensible.

Note that this version deprecates the use of `load_table_model` for XSPEC models.
Sherpa/XSPEC users should use the new `load_xstable_model` function instead.

Details

Infrastructure and minor non-functional changes have been omitted.

242 Avoid use of inspect.getargspec in Python3
Finish off the replacement of inspect.getargspec by inspect.signature.

263 List_data_ids() fails on py3 with mixed id types (Fix 262).
Sherpa was sorting the list of dataset IDs in a non-python3 compliant fashion,
which resulted in issues when using strings and integers together as dataset
IDs. This has now been fixed.

267 add wstat tests
Add several regression tests for wstat.

282 Parallel_map not working on py3 with numcores=1 (Fix 277).
The `utils` function `parallel_map` failed on Python 3 when called with `numcores=1`,
i.e. on systems with only one processor/core. This has been fixed.

283 Sample flux and numpy deprecations (Fix 273 and 276).
The `sample_flux` function was not working under Python 3 if the `scales` argument
was provided. This has been fixed. Also, a `DeprecationWarning` was issued by
`numpy` because during the `sample_flux` execution values were extracted from
an array with non-integer indices. This has also been fixed.

284 String representation of data classes under py3 (Fix 275).
Data classes `DataPHA`, `DataARF`, `DataRMF`, `DataIMG`, and `DataIMGInt` in
`sherpa.astro.data` would throw an exception if users tried to print them as
strings, under Python 3. This has been fixed.

287 Rewrite sherpa.stats.Stat.calc_stat and simplify sherpa.fit.Fit (fix 227 248 289 292).
In order to fix several issues related to the WStat support, and in order to
make the code more maintainable, the `sherpa.stats.Stat.calc_stat` and
`sherpa.fit.Fit` classes have gone through a round of refactoring. This fixes
the following issues: 227 Issues using wstat when grouping/filtering data; 248
backscal column not treated properly for WStat; 289 calc_stat does not error out
if background subtracted data is used with Likelihood statistics; 292 stat info
does not include reduced stat/qval for wstat.

295 Fix display of pileup model in Python 3.5 (Fix 294).
Fix display of instances of `sherpa.astro.models.JDPileup` so that, in Python 3.5,
they can be displayed after the model has been evaluated.

304 replace file -> open (Fix 297).
The `save` and `restore` functions used to use the `file` function which is not
compatible with Python 3. This has now been fixed.

305 Fix python 3 issues with some session commands (Fix 303).
The `set_xlog`, `set_ylog`, and `show_bkg_model` functions were not compatible
with Python 3. This has now been fixed (Issue 303).

307 Move XSPEC table support to load_xstable_model and deprecate its support
in load_table_model (Fix 270).
Add the `load_xstable_model` routine to the `sherpa.astro.ui` module, which
supports loading XSPEC additive or multiplicative (atable and mtable) models.
The support for these models is still available via `load_table_model` in this
release, but it is deprecated. The `read_xstable_model` routine has been added
to the `sherpa.astro.xspec` module.

312 Fix over-zealous code clean up in PR 287 affecting sigmarej.
Fits using the sigmarej iterated-fit method were broken if a filter had been
applied to the data before the fit and there are any bins that get ignored at
larger bin values than the filtered-out data. (This fixes a subtle regression
introduced by 287).

313 Allow sequence=None when using gridsearch and Python 3.5 (Fix 309).
Allow the `gridsearch` optimiser to be used with the `sequence` option set
to `None` for Python 3.5.

Caveats

The requirements for Sherpa are to build with Python 2.7 and 3.5. There has been limited testing with Python 3.6, for which we distribute conda binaries. If in doubt, please install Sherpa in 2.7 or 3.5 environments only. Support for Python versions 3.3 and 3.4 is possible but would require community support.

It has been reported during testing that some versions of the matplotlib conda package do not install properly because of a pyqt v5 dependency. If you encounter this issue, please pin down pyqt to version 4, e.g. `conda install matplotlib pyqt=4`.

The `sherpatest` package is not distributed as a conda package anymore. This will probably be true for the foreseeable future. The `sherpatest` package contains data and functional tests that relies on external datasets, so it allows users and developers to run the entire regression tests suite. If you want to install `sherpatest`, please use pip and github:


$ pip install https://github.com/sherpa/sherpa-test-data/archive/4.9.0.tar.gz


If you decide to run the full regression tests suite you should also have matplotlib installed. If matplotlib is not installed a test will run and fail rather than being skipped. This issue will be fixed in the next release.


ciao4.9
This is a CIAO release.
Binaries will be provided with the next standalone release.
Release notes are collected for all changes since CIAO 4.8, which may be included from multiple standalone releases.

Release Notes

Sherpa 4.9 now runs under both Python 2.7 and Python 3.5. The test infrastructure
has been modified including simplification of the smoke test. Several bug fixes and
enhancements are also included. Specific details are identified below.

14361: Sherpa and Chips wrapper need to replace execfile for python 3 compatibility
This fix modifies the sherpa and chips wrapper scripts to replace the
'execfile' command with an 'exec' command sequence that allows the
wrapper script to be utilized by python 2.7 or python 3.5. Without the
change, the option to specify a command script at the sherpa or chips
command prompt (ie. 'chips /pool1/runme.py') will not work on python 3.5
since the execfile does not exist in python 3.5.

107: Normalize plot labels.
Plots created with `plot_source` used a different format to other
plots when analysis=wavelength, in that LaTeX symbols were used for
Angstrom and lambda (in other plots the string 'Angstrom' is used
instead). The source plots now match the other plots.

138: improve and fix issues in `save_all` function.
- added a new argument to `save_all`: if `outfile` is `None` then the `outfh`
argument is used
to define the output handle (the argument can be any file-like argument, such
as a file
handle like `sys.stdout` or the output of `open`, or a `StringIO` object)
- setting the `clobber` argument to `save_all` now means that the output file
(the `outfile`
argument, if not `None`) is deleted if it already exists; prior to this, the
file would be
appended to instead
- the source expression is now saved correctly for most cases (e.g. when not
using `set_full_model`); this is bug 97 but also affects non-PHA data sets
- the background model expression was not always written out correctly when
using PHA
data sets
- quality and grouping arrays of PHA data sets are now stored as 16-byte
integers rather
than a floating-point value (this has no affect on the results, but matches
the OGIP standard)
- fixed up saving the grouping and quality arrays of background PHA data sets
(this would only
be an issue if the background is being fit, rather than subtracted)
- basic data sets created with the `load_arrays` function are now written out by
`save_all`
as part of the script; this is intended for small datasets and may have
problems with
precision if used with floating-point arrays
- calls to `load_psf` are now correctly restored (they may not have been written
out correctly
if multiple data sets were loaded)
- user models are now written out to disk; this consists of two parts:
- writing out the function that defines the model, which may or may not be
possible (if
not, a place-holder function is added to the output and a warning
displayed).
- the necessary calls to `load_user_model` and `add_user_pars` are now
included in the output
- the Python code created by `save` all has undergone several minor changes:
- it now explicitly imports the `sherpa.astro.ui` module, so that it can be
run from the
IPython prompt using the `%run <filename>` command, or directly as `python <filename>`
- it uses the `create_model_component` function rather than `eval` to create
model
components (this is CXC bug 12146)
- many optional arguments to functions are now given as `name=value` rather
than
being a positional argument, to make it clearer what the script is doing.
- calls to `load_data` have been replaced by more-specific versions - e.g.
`load_pha`
and `load_image` - if appropriate
- there have been several minor syntactic clean ups to better follow the
suggestions from PEP8

When writing out code that defines a user-model, there is no attempt to make
sure that
modules used by the function are available. These will need to be added, either
directly
or imported, manually to the output.

153: Minor bug with calc_chi2datavar_errors (Fix 148).
Make comparison test in `calc_chi2datavar_errors` less stringent, so to
include the case where `sqrt(x)=0`.

155: Add argument to get_draws for supplying a covariance matrix.
The `get_draws` function now accepts a user-provided covariance matrix. If
no covariance matrix is provided, the covariance matrix computed by the default
implementation is used. Note that `covar()` must be invoked before invoking
`get_draws` if no covariance matrix is provided, otherwise `get_draws` will exit
with an error.

165: Remove usage of deprecated numpy API.

185: Protect XPA command to avoid shell confusion.
Fix the problem where if the working directory contained a file called
`x` or `y` then the `sherpa.astro.ui.image_data()` function would fail with the message


DS9Err: Could not display image


187: Issue a more meaningful message when
`sherpa.astro.io` is imported directly and no fits backends are available. (Fix 92).

190: Datastack can be used if no plotter available.
The datastack package can now be used even if there is no
available plotting backend. In this case, plotting functions will not be
available, but the rest of the datastack functionality will. (Fix 22).

195 Generalize calc_stat API + example of how to have several datasets with
different fit statistics. Attempts to generalize the `calc_stat` API and
enable simultaneous fits with different statistics for different data sets.

209: Fix docstrings for `group_snr()` and `group_adapt_snr()`.
Updates to the docstrings for clarity.

210: New Smoke Test
The smoke test has been greatly simplified: rather than running all the unit and regression tests that do not require test data, the smoke test now simply ensures that the basic installation works, i.e. that basic commands can be run and that dependencies can be reached.

211: Cleanup of documentation and code in `sherpa.astro.utils`.
The `calc_kcorr` function is now exported by sherpa.astro.utils. Minor
changes to the documentation in `sherpa.astro.utils` were also made to conform
to Sphinx standards.

221 Add model documentation (Fix 217).
Integrate existing model documentation (from external sources and the CIAO
ahelp documentation system) into the model classes.

229: Code is both Python 2.7 and 3.5 compliant. (Fix 76).

242: Avoid use of `inspect.argspec` in Python 3
This change replaces the deprecated 'inspect.argspec' call with 'inspect.signature'.

252: Fix `plot_photon_flux` function. (Fix 241).
A bug where plotting photon flux was fixed by adding a missing argument to the `sample_photon_flux` call.

253 Make sure background is taken into account in calc_stat_info (Fix 147).
`calc_stat_info` call failed when `wstat` was selected, as the background
was not taken into account. This issue has now been fixed.

254 Fix the documentation for set_rmf (Fix 236).
The documentation for `set_rmf` incorrectly referred to ARF rather than RMF.

256 Fix docstring in set_quality (Fix 205).
The docstring in `set_quality` now correctly indicates the quality flags.
The previous documentation didn't describe the values of such flags properly.

257 Fix docstring for levmar tolerance (Fix 257).
The documentation string for the Levenberg-Marquardt optimization function now
correctly states that the parameter default values are equal to the single
precision epsilon, rather than the square root of the double precision epsilon.

263 List_data_ids() fails on py3 with mixed id types (Fix 262).
Sherpa was sorting the list of dataset IDs in a non-python3 compliant fashion,
which resulted in issues when using strings and integers together as dataset
IDs. This has now been fixed.

267 add wstat tests
Add several regression tests for wstat.

282 Parallel_map not working on py3 with numcores=1 (Fix 277).
The `utils` function `parallel_map` failed on Python 3 when called with `numcores=1`,
i.e. on systems with only one processor/core. This has been fixed.

283 Sample flux and numpy deprecations (Fix 273 and 276).
The `sample_flux` function was not working under Python 3 if the `scales` argument
was provided. This has been fixed. Also, a `DeprecationWarning` was issued by
`numpy` because during the `sample_flux` execution values were extracted from
an array with non-integer indices. This has also been fixed.

284 String representation of data classes under py3 (Fix 275).
Data classes `DataPHA`, `DataARF`, `DataRMF`, `DataIMG`, and `DataIMGInt` in
`sherpa.astro.data` would throw an exception if users tried to print them as
strings, under Python 3. This has been fixed.

287 Rewrite sherpa.stats.Stat.calc_stat and simplify sherpa.fit.Fit (fix 227 248 289 292).
In order to fix several issues related to the WStat support, and in order to
make the code more maintainable, the `sherpa.stats.Stat.calc_stat` and
`sherpa.fit.Fit` classes have gone through a round of refactoring. This fixes
the following issues: 227 Issues using wstat when grouping/filtering data; 248
backscal column not treated properly for WStat; 289 calc_stat does not error out
if background subtracted data is used with Likelihood statistics; 292 stat info
does not include reduced stat/qval for wstat.

295 Fix display of pileup model in Python 3.5 (Fix 294).
Fix display of instances of `sherpa.astro.models.JDPileup` so that, in Python 3.5,
they can be displayed after the model has been evaluated.

304 replace file -> open (Fix 297).
The `save` and `restore` functions used to use the `file` function which is not
compatible with Python 3. This has now been fixed.

305 Fix python 3 issues with some session commands (Fix 303).
The `set_xlog`, `set_ylog`, and `show_bkg_model` functions were not compatible
with Python 3. This has now been fixed (Issue 303).

307 Move XSPEC table support to load_xstable_model and deprecate its support
in load_table_model (Fix 270).
Add the `load_xstable_model` routine to the `sherpa.astro.ui` module, which
supports loading XSPEC additive or multiplicative (atable and mtable) models.
The support for these models is still available via `load_table_model` in this
release, but it is deprecated. The `read_xstable_model` routine has been added
to the `sherpa.astro.xspec` module.

312 Fix over-zealous code clean up in PR 287 affecting sigmarej.
Fits using the sigmarej iterated-fit method were broken if a filter had been
applied to the data before the fit and there are any bins that get ignored at
larger bin values than the filtered-out data. (This fixes a subtle regression
introduced by 287).

313 Allow sequence=None when using gridsearch and Python 3.5 (Fix 309).
Allow the `gridsearch` optimiser to be used with the `sequence` option set
to `None` for Python 3.5.

Caveats

319: `image_getregion` returns byte string on Py3.

SH-2: The new `test_save_restore` test in the CIAO regression tests suite is failing on all platforms. We are investigating the reasons of the failure. The failure is triggered when the test is not run in isolation, and only when certain other tests are run before it. Also, this only applies to CIAO and not to standalone Sherpa.

SH-3: Some tests are skipped during the CIAO regression tests.

SH-4: Several OS X regression tests are failing.

Note: The SH-2/3/4 caveats are issues with the tests themselves, not with the code, and only appear when running the full CIAO regression tests suite. User will not be affected by the above issues unless they run the full CIAO regression tests suite.

This version of Sherpa is the first one to run under both Python 2.7
and Python 3.5. The python 3 conversion is considered a beta designed
to maintain backwards compatibility with python 2.7. As such, we
expect to have some iterations before the Python 3.5 support stabilizes.

The smoke test has been greatly simplified: rather that running all of
the unit and regression tests that do not require test data, the smoke
test now simply ensures that the basic installation works, i.e. that
basic commands can be run and that dependencies can be reached. This
allows the use of more advanced tools for actual unit and regression
tests without having to ship such tools to users.

Several bugs were fixed, some enhancements implemented, and some
deprecated calls to external API replaced. In particular:
- more documentation was migrated and is not part of the code base
as _docstrings_.
- the `calc_stat` API has been generalized to make statistic
functions more extensible.
- `calc_stat_info` now properly takes background into account when
`wstat` is selected.

Details

Infrastructure and minor non-functional changes have been omitted.

209: Fix docstrings for `group_snr()` and `group_adapt_snr()`. Docstrings
used to say "Combine the data so that each bin has a signal-to-noise
ratio of at least minimum" but the end of the groups are marked only
if the SNR exceeds the given SNR.

210: The smoke test has been greatly simplified and it now simply ensures
that the basic installation works, i.e. that basic commands can be run
and that dependencies can be reached. Previously, the smoke test would
run all unit tests. Units and regression tests can now take advantage of
`py.test` and `mock` packages (the latter is part of the standard
library in Python3).

211: Cleanup of documentation and code in `sherpa.astro.utils`. The
`calc_kcorr` function is now exported by sherpa.astro.utils. Minor
changes to the documentation in `sherpa.astro.utils` to conform to
Sphinx standards.

221: Integrate existing model documentation (from external sources and
the CIAO ahelp documentation system) into the model classes.

229: Code is both Python 2.7 and 3.5 compliant. This included updating
the CIAO dependencies: region, group, and stack libraries, which were
ported to Python 3 as well.

242: Avoid use of `inspect.argspec`, which was deprecated in Python3.

252: `plot_photon_flux` function was calling an internal function with
the wrong number of arguments and thus raising an error. This has been
fixed.

253: `calc_stat_info` call failed when `wstat` was selected, as the
background was not taken into account. This issue has now been fixed.

256: Fix docstring in `set_quality`, which were listing the wrong values
of the quality flag.

257: Fix docstring for `levmar` tolerance, which was listing the wrong
function defaults.

Caveats/Known Issues

The following are known issues with the standalone 4.8.2 release

**`show_all` does not work on Python3 with PHA**. E.g., the following
code throws an exception:

python
from sherpa.astro.ui.utils import Session
from sherpa.astro.data import DataPHA
session = Session()
session.load_arrays(1, [1, 2, 3], [1, 2, 3], DataPHA)
session.show_all()


**the `parallel_map` function throws an exception on Python 3** when
only one core is available on the system running Sherpa. This impacts
a number of functions taking advantage of tasks parallelization on
multi-core systems. This functions include `calc_flux`, `sample_flux`,
a number of functions for plotting parameter projections, and the
`grid_search` optimization algorithm.

**string representation of some (but not all) dataset classes is broken
on Python 3**. For instance, the following code will result in an exception:

python
from sherpa.astro.ui import *
load_pha('3c273.pi')
print(get_data())


**`sample_flux` does not work on Python 3 when called with the `scales`
argument**, e.g.:

python
from sherpa.astro import ui

ui.load_pha('sherpa-test-data/sherpatest/3c273.pi')
ui.set_model('polynom1d.p1')
ui.fit()
ui.covar()
scal = ui.get_covar_results().parmaxes
ui.sample_flux(p1, 0.5, 1, num=5, correlated=False, scales=scal)

focused on supporting CIAO 4.8. In particular, this version introduces support
for newer versions of the dependencies, along with some feature enhancements,
bug fixes and additional, more accurate tests.

The newly supported dependencies:
- matplotlib v1.5
- numpy 1.10 and 1.11 (with and without mkl support)
- xspec v12.9.0i (when building from source)
- astropy v1.1.2
- region library v4.8 (from CIAO 4.8)

Please see the Caveats section for known issues regarding the XSpec support.

Mode details below (infrastructure changes are not shown):

102: fix issues when writing out FITS files using the `save_pha` and
`save_table` commands when using the `astropy`/`pyfits` backend (bug 46).
Fix for when the notice2d_id, notice2d_image, and the ignore version functions
are called with an invalid identifier (i.e. an identifier that i snot an integer
or string value). The error is now an ArgumentTypeErr with the message "'ids'
must be an identifier or list of identifiers". It was a NameError with the
message "global name '_argument_type_error' is not defined".

107: Normalize plot labels. There are two main changes for plots of PHA data
sets (and related quantities, such as the source, model, and ARF):
- plots with matplotlib now use the LaTeX support - so that 'cm^2' is
now displayed as a superscript; previously they were displayed
directly. This does not change the display with the ChIPS backend.
- plots created with `plot_source` used a different format to other
plots when analysis=wavelength, in that LaTeX symbols were used for
Angstrom and lambda (in other plots the string 'Angstrom' is used
instead). The source plots now match the other plots.

109: fix 103 and 113 in order to support `matplotlib` v1.5.

116: fix bug 27. The `astropy.io.fits`/`pyfits` interface used deprecated
functionality. The code was updated to use the replacement classes/methods when
available, falling back to the original code if not. The interfaces that were
changed were, when the new symbols are available, to:
- use `astropy.io.fits.BinTableHDU.from_columns` rather than
`astropy.io.fits.new_table`
- use `astropy.io.fits.Header` rather than `astropy.io.CardList`