Climpred

============================

Internals/Minor Fixes
---------------------
- Fix broken GEFS link (:pr:`807`) `Trevor Gamblin`_
- New metric :py:class:`~climpred.metrics._me` Mean Error as ``metric='me'``:
(:issue:`826`, :pr:`827`) `Aaron Spring`_
- Unpin ``xarray`` and ``xclim`` and update package health.
(:issue:`815`, `feedstock-issue <https://github.com/conda-forge/climpred-feedstock/issues/26>`_, :pr:`831`, :pr:`832`) `Aaron Spring`_, `Trevor James Smith`_, `Mathias Hauser`_
- Add Trusted Publishing and leverage GitHub deployment environments. (:issue:`839`, :pr:`840`) `Trevor James Smith`_

============================

.. note::
As both maintainers moved out of academia into industry, this will be probably the last release for a while. If you are interested in maintaining climpred, please ping us.

Bug Fixes
---------
- Fix ``reference="persistence"`` for resampled ``init``.
(:issue:`730`, :pr:`731`) `Aaron Spring`_.
- :py:meth:`.HindcastEnsemble.verify`
``(comparison="m2o", reference="uninitialized", dim="init")``.
(:issue:`735`, :pr:`731`) `Aaron Spring`_.
- :py:meth:`.HindcastEnsemble.remove_bias`
does not drop single item ``lead`` dimension.
(:issue:`771`, :pr:`773`) `Aaron Spring`_.

New Features
------------
- Refactored :py:meth:`.HindcastEnsemble.bootstrap` and
:py:meth:`.PerfectModelEnsemble.bootstrap` based on
:py:meth:`.HindcastEnsemble.verify` and :py:meth:`.PerfectModelEnsemble.verify`,
which makes them more comparable.
``pers_sig`` is removed.
Also ``reference=["climatology", "persistence"]`` skill has variance if
``resample_dim='init'``.
``bootstrap`` relies on either ``set_option(resample_skill_func="...")``:

* ``"loop"``: calls :py:func:`climpred.bootstrap.resample_skill_loop` which loops over iterations and calls ``verify`` every single time. Most understandable and stable, but slow.
* ``"exclude_resample_dim_from_dim"``: calls :py:func:`climpred.bootstrap.resample_skill_exclude_resample_dim_from_dim` which calls ``verify(dim=dim_without_resample_dim)``, resamples over ``resample_dim`` and then takes a mean over ``resample_dim`` if in ``dim``.
Enables
``HindcastEnsemble.bootstrap(resample_dim="init", alignment="same_verifs")``.
Fast alternative for ``resample_dim="init"``.
* ``"resample_before"``: calls :py:func:`climpred.bootstrap.resample_skill_resample_before` which resamples ``iteration`` dimension and then calls ``verify`` vectorized.
Fast alternative for ``resample_dim="member"``.
* ``"default"``: ``climpred`` decides which to use

(relates to :issue:`375`, :pr:`731`) `Aaron Spring`_.
- ``climpred.set_option(resample_skill_func='exclude_resample_dim_from_dim')`` allows
``HindcastEnsemble.bootstrap(alignment='same_verifs', resample_dim='init')``.
Does not work for ``pearson_r``-derived metrics.
(:issue:`582`, :pr:`731`) `Aaron Spring`_.
- :py:func:`climpred.utils.convert_init_lead_to_valid_time_lead` converts ``data(init, lead)``
to ``data(valid_time, lead)`` to visualize predictability barrier and the reverse
:py:func:`climpred.utils.convert_valid_time_lead_to_init_lead`.
(:issue:`774`, :pr:`775`, :pr:`783`) `Aaron Spring`_.

Internals/Minor Fixes
---------------------
- Refactor ``asv`` benchmarking. Add ``run-benchmarks`` label to ``PR`` to run ``asv``
via Github Actions. (:issue:`664`, :pr:`718`) `Aaron Spring`_.
- Remove ``ipython`` from ``requirements.txt``. (:pr:`720`) `Aaron Spring`_.
- Calculating ``np.isin`` on ``asi8`` instead of ``xr.CFTimeIndex`` speeds up
:py:meth:`.HindcastEnsemble.verify` and :py:meth:`.HindcastEnsemble.bootstrap` with
large number of inits. (:issue:`414`, :pr:`724`) `Aaron Spring`_.
- Add option ``bootstrap_resample_skill_func`` for they what skill is resampled in
:py:meth:`.HindcastEnsemble.bootstrap` and
:py:meth:`.PerfectModelEnsemble.bootstrap`, see
:py:class:`~climpred.options.set_options`. (:pr:`731`) `Aaron Spring`_.
- Add option ``resample_iterations_func`` to decide whether
:py:func:`xskillscore.resampling.resample_iterations` or
:py:func:`xskillscore.resampling.resample_iterations` should be used, see
:py:class:`~climpred.options.set_options`. (:pr:`731`) `Aaron Spring`_.
- Add option ``bootstrap_uninitialized_from_iterations_mean`` to exchange
``uninitialized`` skill with the iteration mean ``uninitialized``.
Defaults to False., see :py:class:`~climpred.options.set_options`.
(:pr:`731`) `Aaron Spring`_.
- ``alignment="same_verifs"`` will not result in ``NaN``s in ``valid_time``.
(:pr:`777`) `Aaron Spring`_.
- :py:meth:`.HindcastEnsemble.plot_alignment` ``(return_xr=True)`` contains ``valid_time``
coordinate. (:pr:`779`) `Aaron Spring`_.

Bug Fixes
---------
- Fix ``PerfectModel_persistence_from_initialized_lead_0=True`` with multiple
references. (:issue:`732`, :pr:`733`) `Aaron Spring`_.

Documentation
-------------
- Add `verify dim example <examples/decadal/verify_dim_implications.ipynb>`_ showing
how :py:meth:`.HindcastEnsemble.verify` and :py:meth:`.PerfectModelEnsemble.verify`
are sensitive to ``dim`` and how ``dim`` answers different research questions.
(:pr:`740`) `Aaron Spring`_.

============================

Bug Fixes
---------
- Fix when creating ``valid_time`` from ``lead.attrs["units"]`` in
``["seasons", "years"]`` with multi-month stride in ``init``.
(:issue:`698`, :pr:`700`) `Aaron Spring`_.
- Fix ``seasonality="season"`` in ``reference="climatology"``.
(:issue:`641`, :pr:`703`) `Aaron Spring`_.

New Features
------------
- Upon instantiation, :py:class:`.PredictionEnsemble` generates new
2-dimensional coordinate ``valid_time`` for ``initialized`` from ``init`` and
``lead``, which is matched with ``time`` from ``verification`` during alignment.
(:issue:`575`, :pr:`675`, :pr:`678`) `Aaron Spring`_.

.. :: python

>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST")
>>> hind.lead.attrs["units"] = "years"
>>> climpred.HindcastEnsemble(hind).get_initialized()
<xarray.Dataset>
Dimensions: (lead: 10, member: 10, init: 64)
Coordinates:
* lead (lead) int32 1 2 3 4 5 6 7 8 9 10
* member (member) int32 1 2 3 4 5 6 7 8 9 10
* init (init) object 1954-01-01 00:00:00 ... 2017-01-01 00:00:00
valid_time (lead, init) object 1955-01-01 00:00:00 ... 2027-01-01 00:00:00
Data variables:
SST (init, lead, member) float64 ...

- Allow ``lead`` as ``float`` also if ``calendar="360_day"`` or ``lead.attrs["units"]``
not in ``["years","seasons","months"]``. (:issue:`564`, :pr:`675`) `Aaron Spring`_.
- Implement :py:meth:`.HindcastEnsemble.generate_uninitialized` resampling years
without replacement from ``initialized``. (:issue:`589`, :pr:`591`) `Aaron Spring`_.
- Implement Logarithmic Ensemble Skill Score :py:func:`~climpred.metrics._less`.
(:issue:`239`, :pr:`687`) `Aaron Spring`_.
- :py:meth:`.HindcastEnsemble.remove_seasonality` and
:py:meth:`.PerfectModelEnsemble.remove_seasonality` remove the
seasonality of all ``climpred`` datasets. (:issue:`530`, :pr:`688`) `Aaron Spring`_.
- Add keyword ``groupby`` in :py:meth:`.HindcastEnsemble.verify`,
:py:meth:`.PerfectModelEnsemble.verify`, :py:meth:`.HindcastEnsemble.bootstrap` and
:py:meth:`.PerfectModelEnsemble.bootstrap` to group skill by
initializations seasonality. (:issue:`635`, :pr:`690`) `Aaron Spring`_.


.. :: python

>>> import climpred
>>> hind = climpred.tutorial.load_dataset("NMME_hindcast_Nino34_sst")
>>> obs = climpred.tutorial.load_dataset("NMME_OIv2_Nino34_sst")
>>> hindcast = climpred.HindcastEnsemble(hind).add_observations(obs)
>>> skill for each init month separated
>>> skill = hindcast.verify(
... metric="rmse",
... dim="init",
... comparison="e2o",
... skipna=True,
... alignment="maximize",
... groupby="month",
... )
>>> skill
<xarray.Dataset>
Dimensions: (month: 12, lead: 12, model: 12)
Coordinates:
* lead (lead) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0
* model (model) object 'NCEP-CFSv2' 'NCEP-CFSv1' ... 'GEM-NEMO'
skill <U11 'initialized'
* month (month) int64 1 2 3 4 5 6 7 8 9 10 11 12
Data variables:
sst (month, lead, model) float64 0.4127 0.3837 0.3915 ... 1.255 3.98
>>> skill.sst.plot(hue="model", col="month", col_wrap=3)

- :py:meth:`.HindcastEnsemble.plot_alignment` shows how forecast and
observations are aligned based on the `alignment <alignment.html>`_ keyword.
This may help understanding which dates are matched for the different ``alignment``
approaches. (:issue:`701`, :pr:`702`) `Aaron Spring`_.

.. ipython:: python
:okwarning:
:okexcept:

from climpred.tutorial import load_dataset

hindcast = climpred.HindcastEnsemble(
load_dataset("CESM-DP-SST")
).add_observations(load_dataset("ERSST"))
savefig plot_alignment_example.png width=100%
hindcast.plot_alignment(edgecolor="w")

- Add ``attrs`` to new ``coordinates`` created by ``climpred``.
(:issue:`695`, :pr:`697`) `Aaron Spring`_.
- Add ``seasonality="weekofyear"`` in ``reference="climatology"``.
(:pr:`703`) `Aaron Spring`_.
- Compute ``reference="persistence"`` in
:py:class:`.PerfectModelEnsemble` from ``initialized`` first ``lead``
if :py:class:`~climpred.options.set_options`
``(PerfectModel_persistence_from_initialized_lead_0=True)`` (``False`` by default)
using :py:func:`~climpred.reference.compute_persistence_from_first_lead`.
(:issue:`637`, :pr:`706`) `Aaron Spring`_.


Internals/Minor Fixes
---------------------
- Reduce dependencies. (:pr:`686`) `Aaron Spring`_.
- Add `typing <https://docs.python.org/3/library/typing.html>`_.
(:issue:`685`, :pr:`692`) `Aaron Spring`_.
- refactor ``add_attrs`` into :py:meth:`.HindcastEnsemble.verify` and
:py:meth:`.HindcastEnsemble.bootstrap`. Now all keywords are
captured in the skill dataset attributes ``.attrs``.
(:issue:`475`, :pr:`694`) `Aaron Spring`_.
- docstrings formatting with `blackdocs <https://github.com/keewis/blackdoc>`_.
(:pr:`708`) `Aaron Spring`_.

Documentation
-------------
- Refresh all docs with ``sphinx_book_theme`` and ``myst_nb``.
(:issue:`707`, :pr:`708`, :pr:`709`, :pr:`710`) `Aaron Spring`_.

============================

Adding on to ``v2.1.5``, more bias reduction methods wrapped from
`xclim <https://xclim.readthedocs.io/en/latest/sdba.html>`__
are implemented.

Bug Fixes
---------
- Fix ``results="p"`` in :py:meth:`.HindcastEnsemble.bootstrap` and
:py:meth:`.PerfectModelEnsemble.bootstrap` when
``reference='climatology'``.
(:issue:`668`, :pr:`670`) `Aaron Spring`_.
- :py:meth:`.HindcastEnsemble.remove_bias` for ``how`` in
``["modified_quantile", "basic_quantile", "gamma_mapping", "normal_mapping"]``
from `bias_correction <https://github.com/pankajkarman/bias_correction>`__
takes all ``member`` to create model distribution. (:pr:`667`) `Aaron Spring`_.

New Features
------------
- allow more `bias reduction <bias_removal.html>`_ methods wrapped from
`xclim <https://xclim.readthedocs.io/en/stable/sdba_api.html>`__ in
:py:meth:`.HindcastEnsemble.remove_bias`:

* ``how="EmpiricalQuantileMapping"``:
:py:class:`xclim.sdba.adjustment.EmpiricalQuantileMapping`
* ``how="DetrendedQuantileMapping"``:
:py:class:`xclim.sdba.adjustment.DetrendedQuantileMapping`
* ``how="PrincipalComponents"``:
:py:class:`xclim.sdba.adjustment.PrincipalComponents`
* ``how="QuantileDeltaMapping"``:
:py:class:`xclim.sdba.adjustment.QuantileDeltaMapping`
* ``how="Scaling"``: :py:class:`xclim.sdba.adjustment.Scaling`
* ``how="LOCI"``: :py:class:`xclim.sdba.adjustment.LOCI`

These methods do not respond to ``OPTIONS['seasonality']`` like the other methods.
Provide ``group="init.month"`` to group by month or ``group='init'`` to skip grouping.
Provide ``group=None`` or skip ``group`` to use ``init.{OPTIONS['seasonality']}``.
(:issue:`525`, :pr:`662`, :pr:`666`, :pr:`671`) `Aaron Spring`_.

============================

While ``climpred`` has used in the `ASP summer colloquium 2021 <https://asp.ucar.edu/asp-colloquia>`_, many new features in :py:meth:`.HindcastEnsemble.remove_bias` were implemented.

Breaking changes
----------------
- renamed ``cross_validate`` to ``cv=False`` in
:py:meth:`.HindcastEnsemble.remove_bias`.
Only used when ``train_test_split='unfair-cv'``.
(:issue:`648`, :pr:`655`). `Aaron Spring`_.

Bug Fixes
---------
- Shift back ``init`` by ``lead`` after
:py:meth:`.HindcastEnsemble.verify`.
(:issue:`644`, :pr:`645`) `Aaron Spring`_.

New Features
------------
- :py:meth:`.HindcastEnsemble.remove_bias` accepts new keyword
``train_test_split='fair/unfair/unfair-cv'`` (default ``unfair``) following
`Risbey et al. 2021 <http://www.nature.com/articles/s41467-021-23771-z>`_.
(:issue:`648`, :pr:`655`) `Aaron Spring`_.
- allow more `bias reduction <bias_removal.html>`_ methods in
:py:meth:`.HindcastEnsemble.remove_bias`:

* ``how="additive_mean"``: correcting the mean forecast additively
(already implemented)
* ``how="multiplicative_mean"``: correcting the mean forecast multiplicatively
* ``how="multiplicative_std"``: correcting the standard deviation multiplicatively

Wrapped from `bias_correction <https://github.com/pankajkarman/bias_correction/blob/master/bias_correction.py>`__:

* ``how="modified_quantile"``: `Bai et al. 2016 <https://www.sciencedirect.com/science/article/abs/pii/S0034425716302000?via%3Dihub>`_
* ``how="basic_quantile"``: `Themeßl et al. 2011 <https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1002/joc.2168>`_
* ``how="gamma_mapping"`` and ``how="normal_mapping"``: `Switanek et al. 2017 <https://www.hydrol-earth-syst-sci.net/21/2649/2017/>`_

- :py:meth:`.HindcastEnsemble.remove_bias` now does
`leave-one-out cross validation <https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.LeaveOneOut.html>`_
when passing ``cv='LOO'`` and ``train_test_split='unfair-cv'``.
``cv=True`` falls back to ``cv='LOO'``. (:issue:`643`, :pr:`646`) `Aaron Spring`_.
- Add new metrics :py:func:`~climpred.metrics._spread` and
:py:func:`~climpred.metrics._mul_bias` (:pr:`638`) `Aaron Spring`_.
- Add new tutorial datasets: (:pr:`651`) `Aaron Spring`_.

* ``NMME_OIv2_Nino34_sst`` and ``NMME_hindcast_Nino34_sst`` with monthly leads
* ``Observations_Germany`` and ``ECMWF_S2S_Germany`` with daily leads

- Metadata from `CF convenctions <http://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html>`_
are automatically attached by
`cf_xarray <https://cf-xarray.readthedocs.io/en/latest/generated/xarray.DataArray.cf.add_canonical_attributes.html#xarray.DataArray.cf.add_canonical_attributes>`_.
(:issue:`639`, :pr:`656`) `Aaron Spring`_.
- Raise warning when dimensions ``time``, ``init`` or ``member`` are chunked to show
user how to circumvent ``xskillscore`` chunking ``ValueError`` when passing these
dimensions as ``dim`` in :py:meth:`.HindcastEnsemble.verify` or
:py:meth:`.HindcastEnsemble.bootstrap`.
(:issue:`509`, :pr:`658`) `Aaron Spring`_.
- Implement ``PredictionEnsemble.chunks``. (:pr:`658`) `Aaron Spring`_.

Documentation
-------------
- Speed up `ENSO monthly example <examples/monseas/monthly-enso-subx-example.ipynb>`_
with IRIDL server-side preprocessing
(see `context <https://twitter.com/realaaronspring/status/1406980080883150848?s=21>`_)
(:issue:`594`, :pr:`633`) `Aaron Spring`_.
- Add `CITATION.cff <https://github.com/pangeo-data/climpred/blob/main/CITATION.cff>`_.
Please cite
`Brady and Spring, 2020 <https://joss.theoj.org/papers/10.21105/joss.02781>`_.
(`GH <https://github.com/pangeo-data/climpred/commit/eceb3f46d78c7dd8eb25243b2e0b673ddd78a4b2>`_) `Aaron Spring`_.
- Use ``NMME_OIv2_Nino34_sst`` and ``NMME_hindcast_Nino34_sst`` with monthly leads for
`bias reduction <bias_removal.html>`_ demonstrating
:py:meth:`.HindcastEnsemble.remove_bias`.
(:pr:`646`) `Aaron Spring`_.

============================

New Features
------------
- Allow ``hours``, ``minutes`` and ``seconds`` as ``lead.attrs['units']``.
(:issue:`404`, :pr:`603`) `Aaron Spring`_.
- Allow to set ``seasonality`` via :py:class:`~climpred.options.set_options` to specify
how to group in ``verify(reference='climatology'`` or in
:py:meth:`.HindcastEnsemble.remove_bias`.
(:issue:`529`, :pr:`593`, :pr:`603`) `Aaron Spring`_.
- Allow ``weekofyear`` via ``datetime`` in
:py:meth:`.HindcastEnsemble.remove_bias`, but not yet implemented in
``verify(reference='climatology')``. (:issue:`529`, :pr:`603`) `Aaron Spring`_.
- Allow more dimensions in ``initialized`` than in ``observations``. This is particular
useful if you have forecasts from multiple models (in a ``model`` dimension) and want
to verify against the same observations.
(:issue:`129`, :issue:`528`, :pr:`619`) `Aaron Spring`_.
- Automatically rename dimensions to ``CLIMPRED_ENSEMBLE_DIMS``
[``"init"``, ``"member"``, ``"lead"``] if CF standard_names in coordinate attributes
match: (:issue:`613`, :pr:`622`) `Aaron Spring`_.

* ``"init"``: ``"forecast_reference_time"``
* ``"member"``: ``"realization"``
* ``"lead"``: ``"forecast_period"``
- If ``lead`` coordinate is ``pd.Timedelta``,
:py:class:`.PredictionEnsemble` converts ``lead`` coordinate upon
instantiation to integer ``lead`` and corresponding ``lead.attrs["units"]``.
(:issue:`606`, :pr:`627`) `Aaron Spring`_.
- Require ``xskillscore >= 0.0.20``.
:py:func:`~climpred.metrics._rps` now works with different ``category_edges``
for observations and forecasts, see
`daily ECMWF example <examples/subseasonal/daily-S2S-ECMWF.htmlbiweekly-aggregates>`_.
(:issue:`629`, :pr:`630`) `Aaron Spring`_.
- Set options ``warn_for_failed_PredictionEnsemble_xr_call``,
``warn_for_rename_to_climpred_dims``, ``warn_for_init_coords_int_to_annual``,
``climpred_warnings`` via :py:class:`~climpred.options.set_options`.
(:issue:`628`, :pr:`631`) `Aaron Spring`_.
- :py:class:`.PredictionEnsemble` acts like
:py:class:`xarray.Dataset` and understands ``data_vars``, ``dims``, ``sizes``,
``coords``, ``nbytes``, ``equals``, ``identical``, ``__iter__``, ``__len__``,
``__contains__``, ``__delitem__``. (:issue:`568`, :pr:`632`) `Aaron Spring`_.

Documentation
-------------
- Add `documentation page about publicly available initialized datasets and
corresponding `climpred` examples <initialized-datasets.html>`_.
(:issue:`510`, :issue:`561`, :pr:`600`) `Aaron Spring`_.
- Add `GEFS example <examples/NWP/NWP_GEFS_6h_forecasts.html>`_ for numerical weather
prediction. (:issue:`602`, :pr:`603`) `Aaron Spring`_.
- Add subseasonal `daily ECMWF example <examples/subseasonal/daily-S2S-ECMWF.html>`__
using `climetlab <https://github.com/ecmwf-lab/climetlab-s2s-ai-challenge>`_ to access
hindcasts from ECMWF cloud. (:issue:`587`, :pr:`603`) `Aaron Spring`_.
- Add subseasonal `daily S2S example <examples/subseasonal/daily-S2S-IRIDL.html>`_
accessing `S2S <http://s2sprediction.net/>`_ output on
`IRIDL <https://iridl.ldeo.columbia.edu/SOURCES/.ECMWF/.S2S/>`_ with a cookie and
working with "on-the-fly" reforecasts with ``hdate`` dimension.
(:issue:`588`, :pr:`593`) `Aaron Spring`_.
- Added example `climpred on GPU <examples/misc/climpred_gpu.ipynb>`_. Running
:py:meth:`.PerfectModelEnsemble.verify` on GPU with `cupy-xarray
<https://github.com/xarray-contrib/cupy-xarray>`_ finishes 10x faster.
(:issue:`592`, :pr:`607`) `Aaron Spring`_.
- How to work with biweekly aggregates in ``climpred``, see
`daily ECMWF example <examples/subseasonal/daily-S2S-ECMWF.htmlbiweekly-aggregates>`__.
(:issue:`625`, :pr:`630`) `Aaron Spring`_.


Internals/Minor Fixes
---------------------
- Add weekly upstream CI, which raises issues for failures. Adapted from ``xarray``.
Manually trigger by ``git commit -m '[test-upstream]'``. Skip climpred_testing CI by
``git commit -m '[skip-ci]'``
(:issue:`518`, :pr:`596`) `Aaron Spring`_.