Numpyro

New Features

- New VI inference: [SteinVI](https://num.pyro.ai/en/latest/contrib.html#stein-variational-inference). Checkout a couple examples in PRs 1297 1298 for the usage.
- New distributions: [MultivariateStudentT](https://num.pyro.ai/en/latest/distributions.html#multivariatestudentt), [DiscreteUniform](https://num.pyro.ai/en/latest/distributions.html#discreteuniform), [Kumaraswamy](https://num.pyro.ai/en/latest/distributions.html#kumaraswamy), [RelaxedBernoulli](https://num.pyro.ai/en/latest/distributions.html#relaxedbernoulli) .
- New tutorials and examples:
+ [Tutorial for Truncated distributions](https://num.pyro.ai/en/latest/tutorials/truncated_distributions.html): a complete guide for how to construct a NumPyro distribution.
+ [Bayesian Hierarchical Stacking case study](https://num.pyro.ai/en/latest/tutorials/bayesian_hierarchical_stacking.html) to average models based on weights from a hierarchical structure.
+ [Sine-skewed sine (bivariate von Mises) mixture](https://num.pyro.ai/en/latest/examples/ssbvm_mixture.html) to model the dihedral angles that occur in the backbone of a protein.
+ [AR2 processes](https://num.pyro.ai/en/latest/examples/ar2.html) to show how to avoid the (slow) Python for-loop.
+ [Holt-winter Exponential Smoothing](https://num.pyro.ai/en/latest/examples/holt_winters.html) example for time series forecasting.
+ [Hilbert space approximation for Gaussian processes](https://num.pyro.ai/en/latest/examples/hsgp.html) example is significantly revised.

Enhancements and Bug Fixes

- 1305 Fixes HMCECS bug for likelihoods with multiple plates
- 1304 Improves warning mechanism when plates are missing.
- 1301 Fixes sparse Poisson density sometimes returns int output.
- 1289 Make HMC Gibbs algorithms work with improper distributions
- 1284 Adds various KL divergences for Gamma/Beta families
- 1281 Raises error if there are duplicated deterministic sites
- 1271 Better warning mechanism with stacklevel
- 1270 Incorporate kl divergences of Tensorflow Probability distributions
- 1259 1266 Allow TruncatedNormal/Cauchy to take both low and high
- 1254 `numpyro.contrib.indexing` is moved to `numpyro.ops.indexing`
- 1252 Use multipledispatch for `kl_registry`
- 1250 Added `cdf` methods for gamma, inverse gamma, log normal densities
- 1248 Add ProvenanceArray to infer relational structure in a model
- 1244 Raise warning for the automatic enumeration behavior
- 1237 Enhance warnings for invalid parameters of `BetaProportion` distribution
- 1227 Allow `prior` to be callable in `random_flax_module` and `random_haiku_module`
- 1226 Allow init_to_sample work with scalar values
- 1225 Add color for divergences in Neal's example
- 1196 Allow custom precision function in laplace approximation autoguide
- 1194 Option to specify init state for SVI run
- 1185 1189 Avoid -inf/nan samples in truncated distributions
- 1182 Extend scope handler for plate stack frames
- 1179 Support enumerate support for zero inflated distributions
- 1169 Allow pickle autoguides

This release is composed of great contributions and feedback from the Pyro community: amalvaidya MarcoGorelli omarfsosa maw501 bjeffrey92 hessammehr OlaRonning dykim29 Carlosbogo wataruhashimoto52 Vedranh13 ahmadsalim austereantelope and many others. Thank you!

Breaking changes

Switch to softplus transforms for autoguide scales (thanks to [experiments](https://github.com/pyro-ppl/sandbox/tree/master/2021-03-softplus_scales) performed by vitkl).

New Features

- New autoguide: [AutoDAIS](http://num.pyro.ai/en/latest/autoguide.html#autodais) leverages HMC and annealed importance sampling within a variational inference framework
- New distributions: [MixtureSameFamily](http://num.pyro.ai/en/latest/distributions.html#mixturesamefamily), and directional distributions [SineBivariateVonMises](http://num.pyro.ai/en/latest/distributions.html#sinebivariatevonmises), [SineSkewed](http://num.pyro.ai/en/latest/distributions.html#sineskewed)
- New constraints: [l1_ball](http://num.pyro.ai/en/latest/distributions.html#l1-ball) for vectors with L1 norm less than 1
- New transforms: [L1BallTransform](http://num.pyro.ai/en/latest/distributions.html#l1balltransform), [SimplexToOrderedTransform](http://num.pyro.ai/en/latest/distributions.html#simplextoorderedtransform), [ScaledUnitLowerCholeskyTransform](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.transforms.ScaledUnitLowerCholeskyTransform)
- 1116 New [format_shapes](http://num.pyro.ai/en/latest/utilities.html#numpyro.util.format_shapes) utility to interpret the shapes of random variables/plates in a model.
- 1109 Allow direct use of TFP distributions in numpyro.sample
- New tutorials and examples:
+ [Principled prior with Dirichlet distribution](http://num.pyro.ai/en/latest/tutorials/ordinal_regression.html#Principled-prior-with-Dirichlet-Distribution) for Ordinal Regression case study
+ [Horseshoe regression](http://num.pyro.ai/en/latest/examples/horseshoe_regression.html)
+ [Bad posterior geometry and how to deal with it](http://num.pyro.ai/en/latest/tutorials/bad_posterior_geometry.html)

Enhancements and Bug Fixes

- 1108 Avoid numerical problems when using BernoulliProbs
- 1118 Recommend AutoNormal guide when hessian in AutoLaplace is singular
- 1126 Smarter warning about discrete inference in SVI models
- 1136 Support to use SA sampler with arviz
- 1139 Document Poisson `is_sparse` argument
- 1140 Make Sigmoid and StickBreakingTransform more stable
- 1149 Raise value error if num_steps bad in svi.run
- 1162 Use black[jupyter] in notebooks

This release is composed of great contributions and feedback from the Pyro community: MarcoGorelli OlaRonning d-diaz quattro svilupp peterroelants prashjet freddyaboulton tcbegley julianstastny alexlyttle and many others. Thank you!

This is a patch release with the following new feature and fixes:

+ New example [Hilbert space approximation of Gaussian processes](http://num.pyro.ai/en/stable/examples/hsgp.html) #1097 thanks to omarfsosa
+ Fix for rendering models with only discrete variables 1099 thanks to bdatko
+ Fix progress-bar issues when running multi-chain MCMC 1101

In 0.7.0 release, the wheel file uploaded to PyPI had some files not updated. This release fixes that issue.

Since this release, NumPyro can be installed along with the latest `jax` and `jaxlib` releases (their version restrictions have been relaxed). In addition, NumPyro will use the default JAX platform so if you installed JAX with GPU/TPU support, their devices will be used by default.

New Features

- New distributions: [SoftLaplace](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.SoftLaplace), [Weibull](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.Weibull), [BetaProportion](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.BetaProportion), [NegativeBinomial](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.NegativeBinomial), [NegativeBinomial2](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.NegativeBinomial2), [ZeroInflatedDistribution](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.discrete.ZeroInflatedDistribution), [ZeroInflatedPoisson](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.discrete.ZeroInflatedPoisson), [ZeroInflatedNegativeBinomial2](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.conjugate.ZeroInflatedNegativeBinomial2), [FoldedDistribution](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.distribution.FoldedDistribution)
- Support for DeepMind's [Optax](https://github.com/deepmind/optax) optimizers in [SVI](https://num.pyro.ai/en/latest/svi.html#numpyro.infer.svi.SVI)
- New ELBO objective: [TraceGraph_ELBO](http://num.pyro.ai/en/latest/svi.html#numpyro.infer.elbo.TraceGraph_ELBO) for non-reparameterized latent variables (e.g. discrete latent variables)
- A new wrapper [NestedSampler](https://num.pyro.ai/en/latest/contrib.html#numpyro.contrib.nested_sampling.NestedSampler) to leverage the nested sampling package [jaxns](https://github.com/Joshuaalbert/jaxns) for NumPyro models
- Implement `cdf` and `icdf` methods for many distributions
- New [cond](http://num.pyro.ai/en/latest/primitives.html#numpyro.contrib.control_flow.cond) primitive.
- New [infer_discrete](http://num.pyro.ai/en/latest/funsor.html#numpyro.contrib.funsor.discrete.infer_discrete) handler to sample discrete sites under enumeration. Check out the [annotation example](http://num.pyro.ai/en/latest/examples/annotation.html) for a usage.
- Structural mass matrix can be specified via [dense_mass](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.hmc.NUTS) argument of the HMC/NUTS constructor 963
- New examples:
+ [Thompson sampling for Bayesian optimization with GPs](http://num.pyro.ai/en/latest/examples/thompson_sampling.html)
+ [Latent Dirichlet Allocation for topic modeling](http://num.pyro.ai/en/latest/examples/prodlda.html): a great example to illustrate the usage of Flax/Haiku in NumPyro

Enhancements and Bug Fixes

- Documentation and examples are greatly enhanced to make features more accessible
- Fix chain detection for various CPU device strings 1077
- Fix AutoNormal's `quantiles` method for models with non-scalar latent sites 1066
- Fix LocScaleReparam with `center=1` 1059
- Enhance auto guides to support models with deterministic sites 1022
- Support for mutable states in Flax and Haiku modules 1016
- Fix a bug in auto guides that happens when using the guide in Predictive 1013
- Support decorator syntax for effect handlers 1009
- Implement sparse Poisson log probability 1003
- Support `total_count=0` in Multinomial distribution 1000
- Add a flag to control regularize mass matrix behavior in mass matrix adaptation 998
- Add experimental Dockerfiles 996
- Allow setting max tree depth of NUTS sampler during warmup phase 984
- Fix dimensions mixed up in `ExpandedDistribution.sample` method 972
- MCMC objects can be pickled now 968

This release is made of great contributions and feedbacks from the Pyro community: ahoho, kpj, gustavehug, AndrewCSQ, jatentaki, tcbegley, dominikstrb, justinrporter, dirmeier, irustandi, MarcoGorelli, lumip, and many others. Thank you!

New Features

- Progress bar is available for running parallel MCMC chains.
- New samplers:
+ [BarkerMH](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.barker.BarkerMH) - a Metropolis-Hastings sampler that uses a skew-symmetric proposal distribution that depends on the gradient of the potential
+ New [taylor_proxy](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.hmc_gibbs.HMCECS.taylor_proxy) for [HMCECS](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.hmc_gibbs.HMCECS) sampler. This control variate significantly improves the performance of HMCECS on tall data.
+ [MixedHMC](http://num.pyro.ai/en/latest/mcmc.html#numpyro.infer.mixed_hmc.MixedHMC) for mixed discrete and continuous variables
- New distributions:
+ [ProjectedNormal](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.directional.ProjectedNormal) is similar to von Mises and von Mises-Fisher distributions but permits tractable variational inference via reparametrizers
+ [TruncatedDistribution](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.continuous.TruncatedDistribution) to truncate over a family of symmetric distributions: Cauchy, Laplace, Logistic, Normal, StudentT
- New method [Distribution.infer_shapes()](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.distribution.Distribution.infer_shapes) for static shape analysis.
- New constraints: [sphere](http://num.pyro.ai/en/latest/distributions.html#sphere), [positive_ordered_vector](http://num.pyro.ai/en/latest/distributions.html#positive-ordered-vector), [softplus_positive](http://num.pyro.ai/en/latest/distributions.html#softplus-positive), [softplus_lower_cholesky](http://num.pyro.ai/en/latest/distributions.html#softplus-lower-cholesky)
- New transforms: [SoftplusTransform](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.transforms.SoftplusTransform), [SoftplusLowerCholeskyTransform](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.transforms.SoftplusLowerCholeskyTransform)
- New reparameterizer: [ProjectedNormalReparam](http://num.pyro.ai/en/latest/reparam.html#numpyro.infer.reparam.ProjectedNormalReparam) for `ProjectedNormal` distribution
- New [obs_mask](http://num.pyro.ai/en/latest/primitives.html#numpyro.primitives.sample) argument in `sample` primitive for masked conditioning
- New examples:
+ [Hamiltonian Monte Carlo with Energy Conserving Subsampling](http://num.pyro.ai/en/latest/examples/hmcecs.html)
+ [MCMC Methods for Tall Data](http://num.pyro.ai/en/latest/examples/covtype.html)

Enhancements and Bug Fixes

- Improve precision for Dirichlet distributions with small concentration 943
- Make it easy to use softplus transforms in autoguides 941
- Improving compiling time in MCMC samplers - compiling time is 2x faster than previously 924
- Reduce memory requirement for `AutoLowRankMultivariateNormal.quantiles` 921
- Example of how to use [Distribution.mask](http://num.pyro.ai/en/latest/distributions.html#numpyro.distributions.distribution.Distribution.mask) 917
- Add goodness of fit helpers for testing distributions 916
- Enabling sampling with intermediates for `ExpandedDistribution` 909
- Fix DiscreteHMCGibbs to work with multiple chains 908
- Fix missing `infer` key in `handlers.lift` 892

Thanks loopylangur, Dominik Straub dominikstrb, Jeremie Coullon jeremiecoullon, Ola Rønning OlaRonning, Lukas Prediger lumip, Raúl Peralta Lozada RaulPL, Vitalii Kleshchevnikov vitkl, Matt Ludkin ludkinm, and many others for your contributions and feedback!