================
Release Highlights
------------------
- Hyperspy has split off some of the file reading/writing and domain specific functionalities into separate libraries!
- `RosettaSciIO <https://hyperspy.org/rosettasciio>`_: A library for reading and writing scientific data files.
See `RosettaSciIO release notes <https://hyperspy.org/rosettasciio/changes.html>`_ for new features and supported formats.
- `exSpy <https://exspy.readthedocs.io>`_: A library for EELS and EDS analysis.
See `exSpy release notes <https://hyperspy.org/exspy/changes.html>`_ for new features.
- `HoloSpy <https://holospy.readthedocs.io>`_: A library for analysis of (off-axis) electron holography data.
See `HoloSpy release notes <https://holospy.readthedocs.io/en/latest/changes.html>`_ for new features.
- The :py:mod:`~.api.plot.markers` API has been refactored
- Lazy markers are now supported
- Plotting many markers is now `much` faster
- Added :py:class:`~.api.plot.markers.Polygons` marker
- The documentation has been restructured and improved!
- Short example scripts are now included in the documentation
- Improved guides for lazy computing as well as an improved developer guide
- Plotting is easier and more consistent:
- Added horizontal figure layout choice when using the ``ipympl`` backend
- Changing navigation coordinates using the keyboard arrow-keys has been removed.
Use ``Crtl`` + ``Arrow`` instead.
- Jump to navigation position using ``shift`` + click in the navigator figure.
- HyperSpy now works with Pyodide/Jupyterlite, checkout `hyperspy.org/jupyterlite-hyperspy <https://hyperspy.org/jupyterlite-hyperspy>`_!
- The deprecated API has removed: see the list of API changes and removal in the :ref:`sections below <hyperspy_2.0_api_changes>`.
New features
------------
- :py:meth:`~._signals.lazy.LazySignal.compute` will now pass keyword arguments to the dask :meth:`dask.array.Array.compute` method. This enables setting the scheduler and the number of computational workers. (`2971 <https://github.com/hyperspy/hyperspy/issues/2971>`_)
- Changes to :meth:`~.api.signals.BaseSignal.plot`:
- Added horizontal figure layout choice when using the ``ipympl`` backend. The default layour can be set in the plot section of the preferences GUI. (`3140 <https://github.com/hyperspy/hyperspy/issues/3140>`_)
- Changes to :meth:`~.api.signals.Signal2D.find_peaks`:
- Lazy signals return lazy peak signals
- ``get_intensity`` argument added to get the intensity of the peaks
- The signal axes are now stored in the ``metadata.Peaks.signal_axes`` attribute of the peaks' signal. (`3142 <https://github.com/hyperspy/hyperspy/issues/3142>`_)
- Change the logging output so that logging messages are not displayed in red, to avoid confusion with errors. (`3173 <https://github.com/hyperspy/hyperspy/issues/3173>`_)
- Added ``hyperspy.decorators.deprecated`` and ``hyperspy.decorators.deprecated_argument``:
- Provide consistent and clean deprecation
- Added a guide for deprecating code (`3174 <https://github.com/hyperspy/hyperspy/issues/3174>`_)
- Add functionality to select navigation position using ``shift`` + click in the navigator. (`3175 <https://github.com/hyperspy/hyperspy/issues/3175>`_)
- Added a ``plot_residual`` to :py:meth:`~.models.model1d.Model1D.plot`. When ``True``, a residual line (Signal - Model) appears in the model figure. (`3186 <https://github.com/hyperspy/hyperspy/issues/3186>`_)
- Switch to :meth:`matplotlib.axes.Axes.pcolormesh` for image plots involving non-uniform axes.
The following cases are covered: 2D-signal with arbitrary navigation-dimension, 1D-navigation and 1D-signal (linescan).
Not covered are 2D-navigation images (still uses sliders). (`3192 <https://github.com/hyperspy/hyperspy/issues/3192>`_)
- New :meth:`~.api.signals.BaseSignal.interpolate_on_axis` method to switch one axis of a signal. The data is interpolated in the process. (`3214 <https://github.com/hyperspy/hyperspy/issues/3214>`_)
- Added :func:`~.api.plot.plot_roi_map`. Allows interactively using a set of ROIs to select regions of the signal axes of a signal and visualise how the signal varies in this range spatially. (`3224 <https://github.com/hyperspy/hyperspy/issues/3224>`_)
Bug Fixes
---------
- Improve syntax in the `io` module. (`3091 <https://github.com/hyperspy/hyperspy/issues/3091>`_)
- Fix behaviour of :py:class:`~.misc.utils.DictionaryTreeBrowser` setter with value of dictionary type (`3094 <https://github.com/hyperspy/hyperspy/issues/3094>`_)
- Avoid slowing down fitting by optimising attribute access of model. (`3155 <https://github.com/hyperspy/hyperspy/issues/3155>`_)
- Fix harmless error message when using multiple :class:`~.api.roi.RectangularROI`: check if resizer patches are drawn before removing them. Don't display resizers when adding the widget to the figure (widget in unselected state) for consistency with unselected state (`3222 <https://github.com/hyperspy/hyperspy/issues/3222>`_)
- Fix keeping dtype in :py:meth:`~.api.signals.BaseSignal.rebin` when the endianess is specified in the dtype (`3237 <https://github.com/hyperspy/hyperspy/issues/3237>`_)
- Fix serialization error due to ``traits.api.Property`` not being serializable if a dtype is specified.
See 3261 for more details. (`3262 <https://github.com/hyperspy/hyperspy/issues/3262>`_)
- Fix setting bounds for ``"trf"``, ``"dogbox"`` optimizer (`3244 <https://github.com/hyperspy/hyperspy/issues/3244>`_)
- Fix bugs in new marker implementation:
- Markers str representation fails if the marker isn't added to a signal
- make :meth:`~.api.plot.markers.Markers.from_signal` to work with all markers - it was only working with :class:`~.api.plot.markers.Points` (`3270 <https://github.com/hyperspy/hyperspy/issues/3270>`_)
- Documentation fixes:
- Fix cross-references in documentation and enable sphinx "nitpicky" when building documentation to check for broken links.
- Fix using mutable objects as default argument.
- Change some :class:`~.component.Component` attributes to properties in order to include their docstrings in the API reference. (`3273 <https://github.com/hyperspy/hyperspy/issues/3273>`_)
Improved Documentation
----------------------
- Restructure documentation:
- Improve structure of the API reference
- Improve introduction and overall structure of documentation
- Add gallery of examples (`3050 <https://github.com/hyperspy/hyperspy/issues/3050>`_)
- Add examples to the gallery to show how to use SpanROI and slice signal interactively (`3221 <https://github.com/hyperspy/hyperspy/issues/3221>`_)
- Add a section on keeping a clean and sensible commit history to the developer guide. (`3064 <https://github.com/hyperspy/hyperspy/issues/3064>`_)
- Replace ``sphinx.ext.imgmath`` by ``sphinx.ext.mathjax`` to fix the math rendering in the *ReadTheDocs* build (`3084 <https://github.com/hyperspy/hyperspy/issues/3084>`_)
- Fix docstring examples in :class:`~.api.signals.BaseSignal` class.
Describe how to test docstring examples in developer guide. (`3095 <https://github.com/hyperspy/hyperspy/issues/3095>`_)
- Update intersphinx_mapping links of matplotlib, numpy and scipy. (`3218 <https://github.com/hyperspy/hyperspy/issues/3218>`_)
- Add examples on creating signal from tabular data or reading from a simple text file (`3246 <https://github.com/hyperspy/hyperspy/issues/3246>`_)
- Activate checking of example code in docstring and user guide using ``doctest`` and fix errors in the code. (`3281 <https://github.com/hyperspy/hyperspy/issues/3281>`_)
- Update warning of "beta" state in big data section to be more specific. (`3282 <https://github.com/hyperspy/hyperspy/issues/3282>`_)
Enhancements
------------
- Add support for passing ``**kwargs`` to :py:meth:`~.api.signals.Signal2D.plot` when using heatmap style in :py:func:`~.api.plot.plot_spectra` . (`3219 <https://github.com/hyperspy/hyperspy/issues/3219>`_)
- Add support for pep 660 on editable installs for pyproject.toml based builds of extension (`3252 <https://github.com/hyperspy/hyperspy/issues/3252>`_)
- Make HyperSpy compatible with pyodide (hence JupyterLite):
- Set ``numba`` and ``numexpr`` as optional dependencies.
- Replace ``dill`` by ``cloudpickle``.
- Fallback to dask synchronous scheduler when running on pyodide.
- Reduce packaging size to less than 1MB.
- Add packaging test on GitHub CI. (`3255 <https://github.com/hyperspy/hyperspy/issues/3255>`_)
.. _hyperspy_2.0_api_changes:
API changes
-----------
- RosettaSciIO was split out of the `HyperSpy repository <https://github.com/hyperspy/hyperspy>`_ on July 23, 2022. The IO-plugins and related functions so far developed in HyperSpy were moved to the `RosettaSciIO repository <https://github.com/hyperspy/rosettasciio>`__. (`#2972 <https://github.com/hyperspy/hyperspy/issues/2972>`_)
- Extend the IO functions to accept alias names for format ``name`` as defined in RosettaSciIO. (`3009 <https://github.com/hyperspy/hyperspy/issues/3009>`_)
- Fix behaviour of :meth:`~hyperspy.model.BaseModel.print_current_values`, :meth:`~.component.Component.print_current_values`
and :func:`~.api.print_known_signal_types`, which were not printing when running from a script - they were only printing when running in notebook or qtconsole. Now all print_* functions behave consistently: they all print the output instead of returning an object (string or html). The :func:`IPython.display.display` will pick a suitable rendering when running in an "ipython" context, for example notebook, qtconsole. (`3145 <https://github.com/hyperspy/hyperspy/issues/3145>`_)
- The markers have been refactored - see the new :py:mod:`~.api.plot.markers` API and the :ref:`gallery of examples <gallery.markers>` for usage. The new :py:class:`~.api.plot.markers.Markers` uses :py:class:`matplotlib.collections.Collection`, is faster and more generic than the previous implementation and also supports lazy markers. Markers saved in HyperSpy files (``hspy``, ``zspy``) with HyperSpy < 2.0 are converted automatically when loading the file. (`3148 <https://github.com/hyperspy/hyperspy/issues/3148>`_)
- For all functions with the ``rechunk`` parameter, the default has been changed from ``True`` to ``False``. This means HyperSpy will not automatically try to change the chunking for lazy signals. The old behaviour could lead to a reduction in performance when working with large lazy datasets, for example 4D-STEM data. (`3166 <https://github.com/hyperspy/hyperspy/issues/3166>`_)
- Renamed ``Signal2D.crop_image`` to :meth:`~.api.signals.Signal2D.crop_signal` (`3197 <https://github.com/hyperspy/hyperspy/issues/3197>`_)
- Changes and improvement of the map function:
- Removes the ``parallel`` argument
- Replace the ``max_workers`` with the ``num_workers`` argument to be consistent with ``dask``
- Adds more documentation on setting the dask backend and how to use multiple cores
- Adds ``navigation_chunk`` argument for setting the chunks with a non-lazy signal
- Fix axes handling when the function to be mapped can be applied to the whole dataset - typically when it has the ``axis`` or ``axes`` keyword argument. (`3198 <https://github.com/hyperspy/hyperspy/issues/3198>`_)
- Remove ``physics_tools`` since it is not used and doesn't fit in the scope of HyperSpy. (`3235 <https://github.com/hyperspy/hyperspy/issues/3235>`_)
- Improve the readability of the code by replacing the ``__call__`` method of some objects with the more explicit ``_get_current_data``.
- Rename ``__call__`` method of :py:class:`~.api.signals.BaseSignal` to ``_get_current_data``.
- Rename ``__call__`` method of :py:class:`hyperspy.model.BaseModel` to ``_get_current_data``.
- Remove ``__call__`` method of the :py:class:`hyperspy.component.Component` class. (`3238 <https://github.com/hyperspy/hyperspy/issues/3238>`_)
- Rename ``hyperspy.api.datasets`` to :mod:`hyperspy.api.data` and simplify submodule structure:
- ``hyperspy.api.datasets.artificial_data.get_atomic_resolution_tem_signal2d`` is renamed to :func:`hyperspy.api.data.atomic_resolution_image`
- ``hyperspy.api.datasets.artificial_data.get_luminescence_signal`` is renamed to :func:`hyperspy.api.data.luminescence_signal`
- ``hyperspy.api.datasets.artificial_data.get_wave_image`` is renamed to :func:`hyperspy.api.data.wave_image` (`3253 <https://github.com/hyperspy/hyperspy/issues/3253>`_)
API Removal
-----------
As the HyperSpy API evolves, some of its parts are occasionally reorganized or removed.
When APIs evolve, the old API is deprecated and eventually removed in a major
release. The functions and methods removed in HyperSpy 2.0 are listed below along
with migration advises:
Axes
^^^^
- ``AxesManager.show`` has been removed, use :py:meth:`~.axes.AxesManager.gui` instead.
- ``AxesManager.set_signal_dimension`` has been removed, use :py:meth:`~.api.signals.BaseSignal.as_signal1D`,
:py:meth:`~.api.signals.BaseSignal.as_signal2D` or :py:meth:`~.api.signals.BaseSignal.transpose` of the signal instance instead.
Components
^^^^^^^^^^
- The API of the :py:class:`~.api.model.components1D.Polynomial` has changed (it was deprecated in HyperSpy 1.5). The old API had a single parameters ``coefficients``, which has been replaced by ``a0``, ``a1``, etc.
- The ``legacy`` option (introduced in HyperSpy 1.6) for :class:`~.api.model.components1D.Arctan` has been removed, use :class:`exspy.components.EELSArctan` to use the old API.
- The ``legacy`` option (introduced in HyperSpy 1.6) for :class:`~.api.model.components1D.Voigt` has been removed, use :class:`exspy.components.PESVoigt` to use the old API.
Data Visualization
^^^^^^^^^^^^^^^^^^
- The ``saturated_pixels`` keyword argument of :py:meth:`~.api.signals.Signal2D.plot` has been removed, use ``vmin`` and/or ``vmax`` instead.
- The ``get_complex`` property of ``hyperspy.drawing.signal1d.Signal1DLine`` has been removed.
- The keyword argument ``line_style`` of :py:func:`~.api.plot.plot_spectra` has been renamed to ``linestyle``.
- Changing navigation coordinates using keyboard ``Arrow`` has been removed, use
``Crtl`` + ``Arrow`` instead.
- The ``markers`` submodules can not be imported from the :py:mod:`~.api` anymore, use :py:mod:`hyperspy.api.plot.markers`
directly, i.e. :class:`hyperspy.api.plot.markers.Arrows`, instead.
- The creation of markers has changed to use their class name instead of aliases, for example,
use ``m = hs.plot.markers.Lines`` instead of ``m = hs.plot.markers.line_segment``.
Loading and Saving data
^^^^^^^^^^^^^^^^^^^^^^^
The following deprecated keyword arguments have been removed during the
migration of the IO plugins to the `RosettaSciIO library
<https://hyperspy.org/rosettasciio/changes.html>`_:
- The arguments ``mmap_dir`` and ``load_to_memory`` of the :py:func:`~.api.load`
function have been removed, use the ``lazy`` argument instead.
- :ref:`Bruker composite file (BCF) <bruker-format>`: The ``'spectrum'`` option for the
``select_type`` parameter was removed. Use ``'spectrum_image'`` instead.
- :ref:`Electron Microscopy Dataset (EMD) NCEM <emd_ncem-format>`: Using the
keyword ``dataset_name`` was removed, use ``dataset_path`` instead.
- :ref:`NeXus data format <nexus-format>`: The ``dataset_keys``, ``dataset_paths``
and ``metadata_keys`` keywords were removed. Use ``dataset_key``, ``dataset_path``
and ``metadata_key`` instead.
Machine Learning
^^^^^^^^^^^^^^^^
- The ``polyfit`` keyword argument has been removed. Use ``var_func`` instead.
- The list of possible values for the ``algorithm`` argument of the :py:meth:`~.api.signals.BaseSignal.decomposition` method
has been changed according to the following table:
.. list-table:: Change of the ``algorithm`` argument
:widths: 25 75
:header-rows: 1
* - hyperspy < 2.0
- hyperspy >= 2.0
* - fast_svd
- SVD along with the argument svd_solver="randomized"
* - svd
- SVD
* - fast_mlpca
- MLPCA along with the argument svd_solver="randomized
* - mlpca
- MLPCA
* - nmf
- NMF
* - RPCA_GoDec
- RPCA
- The argument ``learning_rate`` of the ``ORPCA`` algorithm has been renamed to ``subspace_learning_rate``.
- The argument ``momentum`` of the ``ORPCA`` algorithm has been renamed to ``subspace_momentum``.
- The list of possible values for the ``centre`` keyword argument of the :py:meth:`~.api.signals.BaseSignal.decomposition` method
when using the ``SVD`` algorithm has been changed according to the following table:
.. list-table:: Change of the ``centre`` argument
:widths: 50 50
:header-rows: 1
* - hyperspy < 2.0
- hyperspy >= 2.0
* - trials
- navigation
* - variables
- signal
- For lazy signals, a possible value of the ``algorithm`` keyword argument of the
:py:meth:`~._signals.lazy.LazySignal.decomposition` method has been changed
from ``"ONMF"`` to ``"ORNMF"``.
- Setting the ``metadata`` and ``original_metadata`` attribute of signals is removed, use
the :py:meth:`~.misc.utils.DictionaryTreeBrowser.set_item` and
:py:meth:`~.misc.utils.DictionaryTreeBrowser.add_dictionary` methods of the
``metadata`` and ``original_metadata`` attribute instead.
Model fitting
^^^^^^^^^^^^^
- The ``iterpath`` default value has changed from ``'flyback'`` to ``'serpentine'``.
- Changes in the arguments of the :py:meth:`~hyperspy.model.BaseModel.fit` and :py:meth:`~hyperspy.model.BaseModel.multifit` methods:
- The ``fitter`` argument has been renamed to ``optimizer``.
- The list of possible values for the ``optimizer`` argument has been renamed according to the following table:
.. list-table:: Renaming of the ``optimizer`` argument
:widths: 50 50
:header-rows: 1
* - hyperspy < 2.0
- hyperspy >= 2.0
* - fmin
- Nelder-Mead
* - fmin_cg
- CG
* - fmin_ncg
- Newton-CG
* - fmin_bfgs
- Newton-BFGS
* - fmin_l_bfgs_b
- L-BFGS-B
* - fmin_tnc
- TNC
* - fmin_powell
- Powell
* - mpfit
- lm
* - leastsq
- lm
- ``loss_function="ml"`` has been renamed to ``loss_function="ML-poisson"``.
- ``grad=True`` has been changed to ``grad="analytical"``.
- The ``ext_bounding`` argument has been renamed to ``bounded``.
- The ``min_function`` argument has been removed, use the ``loss_function`` argument instead.
- The ``min_function_grad`` argument has been removed, use the ``grad`` argument instead.
- The following :py:class:`~hyperspy.model.BaseModel` methods have been removed:
- ``hyperspy.model.BaseModel.set_boundaries``
- ``hyperspy.model.BaseModel.set_mpfit_parameters_info``
- The arguments ``parallel`` and ``max_workers`` have been removed from the :py:meth:`~hyperspy.model.BaseModel.as_signal` methods.
- Setting the ``metadata`` attribute of a :py:class:`~.samfire.Samfire` has been removed, use
the :py:meth:`~.misc.utils.DictionaryTreeBrowser.set_item` and
:py:meth:`~.misc.utils.DictionaryTreeBrowser.add_dictionary` methods of the
``metadata`` attribute instead.
- The deprecated ``twin_function`` and ``twin_inverse_function`` have been privatized.
- Remove ``fancy`` argument of :meth:`~hyperspy.model.BaseModel.print_current_values` and :meth:`~.component.Component.print_current_values`,
which wasn't changing the output rendering.
- The attribute ``channel_switches`` of :py:class:`~hyperspy.model.BaseModel` have been privatized, instead
use the :py:meth:`~hyperspy.model.BaseModel.set_signal_range_from_mask` or any other methods to
set the signal range, such as :py:meth:`~.models.model1d.Model1D.set_signal_range`,
:py:meth:`~.models.model1d.Model1D.add_signal_range` or :py:meth:`~.models.model1d.Model1D.remove_signal_range`
and their :py:class:`~.models.model2d.Model2D` counterparts.
Signal
^^^^^^
- ``metadata.Signal.binned`` is removed, use the ``is_binned`` axis attribute
instead, e. g. ``s.axes_manager[-1].is_binned``.
- Some possible values for the ``bins`` argument of the :py:meth:`~.api.signals.BaseSignal.get_histogram`
method have been changed according to the following table:
.. list-table:: Change of the ``bins`` argument
:widths: 50 50
:header-rows: 1
* - hyperspy < 2.0
- hyperspy >= 2.0
* - scotts
- scott
* - freedman
- fd
- The ``integrate_in_range`` method has been removed, use :py:class:`~.roi.SpanROI`
followed by :py:meth:`~.api.signals.BaseSignal.integrate1D` instead.
- The ``progressbar`` keyword argument of the :py:meth:`~._signals.lazy.LazySignal.compute` method
has been removed, use ``show_progressbar`` instead.
- The deprecated ``comp_label`` argument of the methods :py:meth:`~.api.signals.BaseSignal.plot_decomposition_loadings`,
:py:meth:`~.api.signals.BaseSignal.plot_decomposition_factors`, :py:meth:`~.api.signals.BaseSignal.plot_bss_loadings`,
:py:meth:`~.api.signals.BaseSignal.plot_bss_factors`, :py:meth:`~.api.signals.BaseSignal.plot_cluster_distances`,
:py:meth:`~.api.signals.BaseSignal.plot_cluster_labels` has been removed, use the ``title`` argument instead.
- The :py:meth:`~.api.signals.BaseSignal.set_signal_type` now raises an error when passing
``None`` to the ``signal_type`` argument. Use ``signal_type=""`` instead.
- Passing an "iterating over navigation argument" to the :py:meth:`~.api.signals.BaseSignal.map`
method is removed, pass a HyperSpy signal with suitable navigation and signal shape instead.
Signal2D
^^^^^^^^
- :meth:`~.api.signals.Signal2D.find_peaks` now returns lazy signals in case of lazy input signal.
Preferences
^^^^^^^^^^^
- The ``warn_if_guis_are_missing`` HyperSpy preferences setting has been removed,
as it is not necessary anymore.
Maintenance
-----------
- Pin third party GitHub actions and add maintenance guidelines on how to update them (`3027 <https://github.com/hyperspy/hyperspy/issues/3027>`_)
- Drop support for python 3.7, update oldest supported dependencies and simplify code accordingly (`3144 <https://github.com/hyperspy/hyperspy/issues/3144>`_)
- IPython and IParallel are now optional dependencies (`3145 <https://github.com/hyperspy/hyperspy/issues/3145>`_)
- Fix Numpy 1.25 deprecation: implicit array to scalar conversion in :py:meth:`~.signals.Signal2D.align2D` (`3189 <https://github.com/hyperspy/hyperspy/issues/3189>`_)
- Replace deprecated :mod:`scipy.misc` by :mod:`scipy.datasets` in documentation (`3225 <https://github.com/hyperspy/hyperspy/issues/3225>`_)
- Fix documentation version switcher (`3228 <https://github.com/hyperspy/hyperspy/issues/3228>`_)
- Replace deprecated :py:class:`scipy.interpolate.interp1d` with :py:func:`scipy.interpolate.make_interp_spline` (`3233 <https://github.com/hyperspy/hyperspy/issues/3233>`_)
- Add support for python 3.12 (`3256 <https://github.com/hyperspy/hyperspy/issues/3256>`_)
- Consolidate package metadata:
- use ``pyproject.toml`` only
- clean up unmaintained packaging files
- use ``setuptools_scm`` to define version
- add python 3.12 to test matrix (`3268 <https://github.com/hyperspy/hyperspy/issues/3268>`_)
- Pin pytest-xdist to 3.5 as a workaround for test suite failure on Azure Pipeline (`3274 <https://github.com/hyperspy/hyperspy/issues/3274>`_)
.. _changes_1.7.6: