Shap

This release incorporates many changes that were originally contributed by the SHAP community via dsgibbons's [Community Fork][fork], which has now been merged into the main shap repository. PRs from this origin are labelled here as `fork123`.

This will be the last release that supports python 3.7.

[fork]: https://github.com/slundberg/shap/discussions/2942

Added

- Added support for python 3.11 ([fork72](https://github.com/dsgibbons/shap/pull/72) by connortann).
- Added `n_points` parameter to all functions in `shap.datasets` ([fork39](https://github.com/dsgibbons/shap/pull/39) by thatlittleboy).
- Added `__call__` to `KernelExplainer` ([2966](https://github.com/slundberg/shap/pull/2966) by dwolfeu).
- Added [contributing guidelines][contrib-guide] ([2996](https://github.com/slundberg/shap/pull/2996) by connortann).

[contrib-guide]: [https://github.com/slundberg/shap/blob/master/CONTRIBUTING.md]

Fixed

- Fixed `plot.waterfall` to support yticklabels with boolean features ([fork58](https://github.com/dsgibbons/shap/pull/58) by dwolfeu).
- Prevent `TreeExplainer.__call__` from throwing ValueError when passed a pandas DataFrame containing Categorical columns ([fork88](https://github.com/dsgibbons/shap/pull/88) by thatlittleboy).
- Fixed sampling in `shap.datasets` to sample without replacement ([fork36](https://github.com/dsgibbons/shap/pull/36) by thatlittleboy).
- Fixed an `UnboundLocalError` problem arising from passing a dictionary input to `shap.plots.bar` ([3001](https://github.com/slundberg/shap/pull/3000) by thatlittleboy).
- Fixed tensorflow import issue with Pyspark when using `Gradient` ([2983](https://github.com/slundberg/shap/pull/2983) by skamdar).
- Fixed the aspect ratio of the colorbar in `shap.plots.heatmap`, and use the `ax` matplotlib API internally for plotting ([3040](https://github.com/slundberg/shap/pull/3040) by thatlittleboy).
- Fixed deprecation warnings for `numba>=0.44` ([fork9](https://github.com/dsgibbons/shap/pull/9) and [fork#68](https://github.com/dsgibbons/shap/pull/68) by connortann).
- Fixed deprecation warnings for `numpy>=1.24` from numpy types ([fork7](https://github.com/dsgibbons/shap/pull/7) by dsgibbons).
- Fixed deprecation warnings for `Ipython>=8` from `Ipython.core.display` ([fork13](https://github.com/dsgibbons/shap/pull/13) by thatlittleboy).
- Fixed deprecation warnings for `tensorflow>=2.11` from `tf.optimisers` ([fork16](https://github.com/dsgibbons/shap/pull/16) by simonangerbauer).
- Fixed deprecation warnings for `sklearn>=1.2` from `sklearn.linear_model` ([fork22](https://github.com/dsgibbons/shap/pull/22) by dsgibbons).
- Fixed deprecation warnings for `xgboost>=1.4` from `ntree_limit` in tree explainer ([2987](https://github.com/slundberg/shap/pull/2987) by adnene-guessoum).
- Fixed build on Windows and MacOS ([3015](https://github.com/slundberg/shap/pull/3015) by PrimozGodec; [#3028](https://github.com/slundberg/shap/pull/3028), [#3029](https://github.com/slundberg/shap/pull/3029) and [#3031](https://github.com/slundberg/shap/pull/3031) by connortann).
- Fixed creation of ragged arrays in `shap.explainers.Exact` ([3064](https://github.com/slundberg/shap/pull/3064) by connortann).

Changed

- Updates to docstrings of several `shap.plots` functions ([3003](https://github.com/slundberg/shap/pull/3003), [#3005](https://github.com/slundberg/shap/pull/3005) by thatlittleboy).

Removed

- Deprecated the Boston house price dataset ([fork38](https://github.com/dsgibbons/shap/pull/38) by thatlittleboy).
- Removed the unused `mimic.py` file and `MimicExplainer` code ([fork53](https://github.com/dsgibbons/shap/pull/53) by thatlittleboy).

Maintenance

- Fixed failing unit tests ([fork29](https://github.com/dsgibbons/shap/pull/29) by dsgibbons, [fork#20](https://github.com/dsgibbons/shap/pull/20) by simonangerbauer, [#3044](https://github.com/slundberg/shap/pull/3044) and [fork#24](https://github.com/dsgibbons/shap/pull/24) by connortann).
- Include CUDA GPU C extension files in the source distribution ([3009](https://github.com/slundberg/shap/pull/3009) by jklaise).
- Fixed installation of package via setuptools ([fork51](https://github.com/dsgibbons/shap/pull/51) by thatlittleboy).
- Introduced a minimal set of `ruff` linting ([fork25](https://github.com/dsgibbons/shap/pull/25), [fork#26](https://github.com/dsgibbons/shap/pull/26), [fork#27](https://github.com/dsgibbons/shap/pull/27), [#2973](https://github.com/slundberg/shap/pull/2973), [#2972](https://github.com/slundberg/shap/pull/2972) and [#2976](https://github.com/slundberg/shap/pull/2976) by connortann; [#2968](https://github.com/slundberg/shap/pull/2968), [#2986](https://github.com/slundberg/shap/pull/2986) by thatlittleboy).
- Updated project metadata to PEP 517 ([3022](https://github.com/slundberg/shap/pull/3022) by connortann).
- Introduced more thorough testing on CI against newer dependencies ([fork61](https://github.com/dsgibbons/shap/pull/61) and [#3017](https://github.com/slundberg/shap/pull/3017) by connortann)
- Reduced unit test time by ~5 mins ([3046](https://github.com/slundberg/shap/pull/3046) by connortann).
- Introduced fixtures for reproducible fuzz testing ([3048](https://github.com/slundberg/shap/pull/3048) by connortann).

Lots of bugs fixes and API improvements.

- Fixed rare bug with XGBoost model loading by TheZL lrjball
- Fixed the beeswarm plot so it does not modify the passed explanation object, ravwojdyla
- Automatic wheel building using GH actions by quantumtec
- GC collection for memory in KernelExplainer by Qingtian-Zou
- Fixed max_evals params for PartitionExplainer
- JIT optimize the PartitionExplainer
- Fix colorbar formatting issues SleepyPepperHead
- New benchmark notebooks
- Use display_data for plotting when possible yuuuxt
- Improved GPUTreeShap compilation and params RAMitchell
- Fix TF API change in DeepExplainer filusn
- Add torch tensor support for plots alexander-pv
- Switch to Github actions for testing instead of Travis
- New California demo dataset swalsh1123
- Fix waterfall plot bug RichardScottOZ
- Handle missing matplotlib installation klieret
- Add linearize link support for Additive explainer (Nandish Gupta)
- Fix exceptions to be more specific alexisdrakopoulos collinb9
- Add color map option for plotting tlabarta
- Release fixed numpy version requirement rmehyde
- And many other contributions kindly made by WeichenXu123 imatiach-msft zeshengli nkthiebaut songololo GiovannaNicora joshzwiebel Ashishbodla navdeep-G smathewmanuel ycouble anubhavmaity adityasaini70 ngupta20 jckkvs abs428 JulesCollenne Tiagosf00 javirandor and Thuener

This release contains many bugs fixes and lots of new functionality, specifically for transformer based NLP models. Some highlights include:
- New plots, bug fixes, docs, and features for NLP model explanations (see docs for details).
- important permutation explainer performance fix by sander-sn
- New joint scatter plots to plot many at once on the same y-scale
- better tree model memory usage by morriskurz
- new docs by coryroyce
- new wheel building by PrimozGodec
- dark mode improvements for the docs by gialmisi
- api tweaks by c56pony nsorros jebarb

Lots of new text explainer work courtesy of ryserrao and serialization courtesy of vivekchettiar! (will note all the other changes later)

Fixes a version mismatch with the v0.38.0 release and serialization updates.

This release contains improved support for explanations of transformer text models and support for the new Explanation object based API. Specific improvements include:

- Transformer model support in the Text explainer courtesy of ryserrao
- Interventional Tree explainer GPU support courtesy of RAMitchell
- Image captioning model support courtesy of anusham1990
- Benchmarking improvements courtesy of maggiewu19
- New text and image visualizations courtesy of vivekchettiar
- New explainer serialization support courtesy of vivekchettiar
- Bug fixes for Linear explainer and the new API courtesy of heimengqi
- Fix for categorical plots courtesy of jeffreyftang
- CUDA support improvements courtesy of JohnZed
- Support for econML model courtesy of vasilismsr
- Many other bug fixes and API improvements.