many new features, numerous bug-fixes, improved test coverage and better
documentation. There have been a number of deprecations and API changes
in this release, which are documented below. All users are encouraged to
upgrade to this release, as there are a large number of bug-fixes and
optimizations. Before upgrading, we recommend that users check that
their own code does not use deprecated SciPy functionality (to do so,
run your code with ``python -Wd`` and check for ``DeprecationWarning`` s).
Our development attention will now shift to bug-fix releases on the
1.7.x branch, and on adding new features on the master branch.
This release requires Python `3.7+` and NumPy `1.16.5` or greater.
For running on PyPy, PyPy3 `6.0+` is required.
Highlights of this release
- A new submodule for quasi-Monte Carlo, `scipy.stats.qmc`, was added
- The documentation design was updated to use the same PyData-Sphinx theme as
other NumFOCUS packages like NumPy.
- We now vendor and leverage the Boost C++ library to enable numerous
improvements for long-standing weaknesses in `scipy.stats`
- `scipy.stats` has six new distributions, eight new (or overhauled)
hypothesis tests, a new function for bootstrapping, a class that enables
fast random variate sampling and percentile point function evaluation,
and many other enhancements.
- ``cdist`` and ``pdist`` distance calculations are faster for several metrics,
especially weighted cases, thanks to a rewrite to a new C++ backend framework
- A new class for radial basis function interpolation, `RBFInterpolator`, was
added to address issues with the `Rbf` class.
*We gratefully acknowledge the Chan-Zuckerberg Initiative Essential Open Source
Software for Science program for supporting many of the improvements to*
`scipy.stats`.
New features
`scipy.cluster` improvements
An optional argument, ``seed``, has been added to ``kmeans`` and ``kmeans2`` to
set the random generator and random state.
`scipy.interpolate` improvements
Improved input validation and error messages for ``fitpack.bispev`` and
``fitpack.parder`` for scenarios that previously caused substantial confusion
for users.
The class `RBFInterpolator` was added to supersede the `Rbf` class. The new
class has usage that more closely follows other interpolator classes, corrects
sign errors that caused unexpected smoothing behavior, includes polynomial
terms in the interpolant (which are necessary for some RBF choices), and
supports interpolation using only the k-nearest neighbors for memory
efficiency.
`scipy.linalg` improvements
An LAPACK wrapper was added for access to the ``tgexc`` subroutine.
`scipy.ndimage` improvements
`scipy.ndimage.affine_transform` is now able to infer the ``output_shape`` from
the ``out`` array.
`scipy.optimize` improvements
The optional parameter ``bounds`` was added to
``_minimize_neldermead`` to support bounds constraints
for the Nelder-Mead solver.
``trustregion`` methods ``trust-krylov``, ``dogleg`` and ``trust-ncg`` can now
estimate ``hess`` by finite difference using one of
``["2-point", "3-point", "cs"]``.
``halton`` was added as a ``sampling_method`` in `scipy.optimize.shgo`.
``sobol`` was fixed and is now using `scipy.stats.qmc.Sobol`.
``halton`` and ``sobol`` were added as ``init`` methods in
`scipy.optimize.differential_evolution.`
``differential_evolution`` now accepts an ``x0`` parameter to provide an
initial guess for the minimization.
``least_squares`` has a modest performance improvement when SciPy is built
with Pythran transpiler enabled.
When ``linprog`` is used with ``method`` ``'highs'``, ``'highs-ipm'``, or
``'highs-ds'``, the result object now reports the marginals (AKA shadow
prices, dual values) and residuals associated with each constraint.
`scipy.signal` improvements
``get_window`` supports ``general_cosine`` and ``general_hamming`` window
functions.
`scipy.signal.medfilt2d` now releases the GIL where appropriate to enable
performance gains via multithreaded calculations.
`scipy.sparse` improvements
Addition of ``dia_matrix`` sparse matrices is now faster.
`scipy.spatial` improvements
``distance.cdist`` and ``distance.pdist`` performance has greatly improved for
certain weighted metrics. Namely: ``minkowski``, ``euclidean``, ``chebyshev``,
``canberra``, and ``cityblock``.
Modest performance improvements for many of the unweighted ``cdist`` and
``pdist`` metrics noted above.
The parameter ``seed`` was added to `scipy.spatial.vq.kmeans` and
`scipy.spatial.vq.kmeans2`.
The parameters ``axis`` and ``keepdims`` where added to
`scipy.spatial.distance.jensenshannon`.
The ``rotation`` methods ``from_rotvec`` and ``as_rotvec`` now accept a
``degrees`` argument to specify usage of degrees instead of radians.
`scipy.special` improvements
Wright's generalized Bessel function for positive arguments was added as
`scipy.special.wright_bessel.`
An implementation of the inverse of the Log CDF of the Normal Distribution is
now available via `scipy.special.ndtri_exp`.
`scipy.stats` improvements
Hypothesis Tests
The Mann-Whitney-Wilcoxon test, ``mannwhitneyu``, has been rewritten. It now
supports n-dimensional input, an exact test method when there are no ties,
and improved documentation. Please see "Other changes" for adjustments to
default behavior.
The new function `scipy.stats.binomtest` replaces `scipy.stats.binom_test`. The
new function returns an object that calculates a confidence intervals of the
proportion parameter. Also, performance was improved from O(n) to O(log(n)) by
using binary search.
The two-sample version of the Cramer-von Mises test is implemented in
`scipy.stats.cramervonmises_2samp`.
The Alexander-Govern test is implemented in the new function
`scipy.stats.alexandergovern`.
The new functions `scipy.stats.barnard_exact` and `scipy.stats. boschloo_exact`
respectively perform Barnard's exact test and Boschloo's exact test
for 2x2 contingency tables.
The new function `scipy.stats.page_trend_test` performs Page's test for ordered
alternatives.
The new function `scipy.stats.somersd` performs Somers' D test for ordinal
association between two variables.
An option, ``permutations``, has been added in `scipy.stats.ttest_ind` to
perform permutation t-tests. A ``trim`` option was also added to perform
a trimmed (Yuen's) t-test.
The ``alternative`` parameter was added to the ``skewtest``, ``kurtosistest``,
``ranksums``, ``mood``, ``ansari``, ``linregress``, and ``spearmanr`` functions
to allow one-sided hypothesis testing.
Sample statistics
The new function `scipy.stats.differential_entropy` estimates the differential
entropy of a continuous distribution from a sample.
The ``boxcox`` and ``boxcox_normmax`` now allow the user to control the
optimizer used to minimize the negative log-likelihood function.
A new function `scipy.stats.contingency.relative_risk` calculates the
relative risk, or risk ratio, of a 2x2 contingency table. The object
returned has a method to compute the confidence interval of the relative risk.
Performance improvements in the ``skew`` and ``kurtosis`` functions achieved
by removal of repeated/redundant calculations.
Substantial performance improvements in `scipy.stats.mstats.hdquantiles_sd`.
The new function `scipy.stats.contingency.association` computes several
measures of association for a contingency table: Pearsons contingency
coefficient, Cramer's V, and Tschuprow's T.
The parameter ``nan_policy`` was added to `scipy.stats.zmap` to provide options
for handling the occurrence of ``nan`` in the input data.
The parameter ``ddof`` was added to `scipy.stats.variation` and
`scipy.stats.mstats.variation`.
The parameter ``weights`` was added to `scipy.stats.gmean`.
Statistical Distributions
We now vendor and leverage the Boost C++ library to address a number of
previously reported issues in ``stats``. Notably, ``beta``, ``binom``,
``nbinom`` now have Boost backends, and it is straightforward to leverage
the backend for additional functions.
The skew Cauchy probability distribution has been implemented as
`scipy.stats.skewcauchy`.
The Zipfian probability distribution has been implemented as
`scipy.stats.zipfian`.
The new distributions ``nchypergeom_fisher`` and ``nchypergeom_wallenius``
implement the Fisher and Wallenius versions of the noncentral hypergeometric
distribution, respectively.
The generalized hyperbolic distribution was added in
`scipy.stats.genhyperbolic`.
The studentized range distribution was added in `scipy.stats.studentized_range`.
`scipy.stats.argus` now has improved handling for small parameter values.
Better argument handling/preparation has resulted in performance improvements
for many distributions.
The ``cosine`` distribution has added ufuncs for ``ppf``, ``cdf``, ``sf``, and
``isf`` methods including numerical precision improvements at the edges of the
support of the distribution.
An option to fit the distribution to data by the method of moments has been
added to the ``fit`` method of the univariate continuous distributions.
Other
`scipy.stats.bootstrap` has been added to allow estimation of the confidence
interval and standard error of a statistic.
The new function `scipy.stats.contingency.crosstab` computes a contingency
table (i.e. a table of counts of unique entries) for the given data.
`scipy.stats.NumericalInverseHermite` enables fast random variate sampling
and percentile point function evaluation of an arbitrary univariate statistical
distribution.
New `scipy.stats.qmc` module
This new module provides Quasi-Monte Carlo (QMC) generators and associated
helper functions.
It provides a generic class `scipy.stats.qmc.QMCEngine` which defines a QMC
engine/sampler. An engine is state aware: it can be continued, advanced and
reset. 3 base samplers are available:
- `scipy.stats.qmc.Sobol` the well known Sobol low discrepancy sequence.
Several warnings have been added to guide the user into properly using this
sampler. The sequence is scrambled by default.
- `scipy.stats.qmc.Halton`: Halton low discrepancy sequence. The sequence is
scrambled by default.
- `scipy.stats.qmc.LatinHypercube`: plain LHS design.
And 2 special samplers are available:
- `scipy.stats.qmc.MultinomialQMC`: sampling from a multinomial distribution
using any of the base `scipy.stats.qmc.QMCEngine`.
- `scipy.stats.qmc.MultivariateNormalQMC`: sampling from a multivariate Normal
using any of the base `scipy.stats.qmc.QMCEngine`.
The module also provide the following helpers:
- `scipy.stats.qmc.discrepancy`: assess the quality of a set of points in terms
of space coverage.
- `scipy.stats.qmc.update_discrepancy`: can be used in an optimization loop to
construct a good set of points.
- `scipy.stats.qmc.scale`: easily scale a set of points from (to) the unit
interval to (from) a given range.
Deprecated features
`scipy.linalg` deprecations
- `scipy.linalg.pinv2` is deprecated and its functionality is completely
subsumed into `scipy.linalg.pinv`
- Both ``rcond``, ``cond`` keywords of `scipy.linalg.pinv` and
`scipy.linalg.pinvh` were not working and now are deprecated. They are now
replaced with functioning ``atol`` and ``rtol`` keywords with clear usage.
`scipy.spatial` deprecations
- `scipy.spatial.distance` metrics expect 1d input vectors but will call
``np.squeeze`` on their inputs to accept any extra length-1 dimensions. That
behaviour is now deprecated.
Backwards incompatible changes
Other changes
We now accept and leverage performance improvements from the ahead-of-time
Python-to-C++ transpiler, Pythran, which can be optionally disabled (via
``export SCIPY_USE_PYTHRAN=0``) but is enabled by default at build time.
There are two changes to the default behavior of `scipy.stats.mannwhitenyu`:
- For years, use of the default ``alternative=None`` was deprecated; explicit
``alternative`` specification was required. Use of the new default value of
``alternative``, "two-sided", is now permitted.
- Previously, all p-values were based on an asymptotic approximation. Now, for
small samples without ties, the p-values returned are exact by default.
Support has been added for PEP 621 (project metadata in ``pyproject.toml``)
We now support a Gitpod environment to reduce the barrier to entry for SciPy
development; for more details see `quickstart-gitpod`.
Authors
* endolith
* Jelle Aalbers +
* Adam +
* Tania Allard +
* Sven Baars +
* Max Balandat +
* baumgarc +
* Christoph Baumgarten
* Peter Bell
* Lilian Besson
* Robinson Besson +
* Max Bolingbroke
* Blair Bonnett +
* Jordão Bragantini
* Harm Buisman +
* Evgeni Burovski
* Matthias Bussonnier
* Dominic C
* CJ Carey
* Ramón Casero +
* Chachay +
* charlotte12l +
* Benjamin Curtice Corbett +
* Falcon Dai +
* Ian Dall +
* Terry Davis
* droussea2001 +
* DWesl +
* dwight200 +
* Thomas J. Fan +
* Joseph Fox-Rabinovitz
* Max Frei +
* Laura Gutierrez Funderburk +
* gbonomib +
* Matthias Geier +
* Pradipta Ghosh +
* Ralf Gommers
* Evan H +
* h-vetinari
* Matt Haberland
* Anselm Hahn +
* Alex Henrie
* Piet Hessenius +
* Trever Hines +
* Elisha Hollander +
* Stephan Hoyer
* Tom Hu +
* Kei Ishikawa +
* Julien Jerphanion
* Robert Kern
* Shashank KS +
* Peter Mahler Larsen
* Eric Larson
* Cheng H. Lee +
* Gregory R. Lee
* Jean-Benoist Leger +
* lgfunderburk +
* liam-o-marsh +
* Xingyu Liu +
* Alex Loftus +
* Christian Lorentzen +
* Cong Ma
* Marc +
* MarkPundurs +
* Markus Löning +
* Liam Marsh +
* Nicholas McKibben
* melissawm +
* Jamie Morton
* Andrew Nelson
* Nikola Forró
* Tor Nordam +
* Olivier Gauthé +
* Rohit Pandey +
* Avanindra Kumar Pandeya +
* Tirth Patel
* paugier +
* Alex H. Wagner, PhD +
* Jeff Plourde +
* Ilhan Polat
* pranavrajpal +
* Vladyslav Rachek
* Bharat Raghunathan
* Recursing +
* Tyler Reddy
* Lucas Roberts
* Gregor Robinson +
* Pamphile Roy +
* Atsushi Sakai
* Benjamin Santos
* Martin K. Scherer +
* Thomas Schmelzer +
* Daniel Scott +
* Sebastian Wallkötter +
* serge-sans-paille +
* Namami Shanker +
* Masashi Shibata +
* Alexandre de Siqueira +
* Albert Steppi +
* Adam J. Stewart +
* Kai Striega
* Diana Sukhoverkhova
* Søren Fuglede Jørgensen
* Mike Taves
* Dan Temkin +
* Nicolas Tessore +
* tsubota20 +
* Robert Uhl
* christos val +
* Bas van Beek +
* Ashutosh Varma +
* Jose Vazquez +
* Sebastiano Vigna
* Aditya Vijaykumar
* VNMabus
* Arthur Volant +
* Samuel Wallan
* Stefan van der Walt
* Warren Weckesser
* Anreas Weh
* Josh Wilson
* Rory Yorke
* Egor Zemlyanoy
* Marc Zoeller +
* zoj613 +
* 秋纫 +
A total of 126 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.
This list of names is automatically generated, and may not be fully complete.