Thinc

> The main new feature of Thinc v9 is the support for [learning rate schedules](https://thinc.ai/docs/api-schedules#schedule) that can take the training dynamics into account. For example, the new [`plateau.v1`](https://thinc.ai/docs/api-schedules#plateau) schedule scales the learning rate when no progress has been found after a given number of evaluation steps. Another visible change is that [`AppleOps`](https://thinc.ai/docs/api-backends) is now part of Thinc, so it is not necessary anymore to install [`thinc-apple-ops`](https://github.com/explosion/thinc-apple-ops) to use the AMX units on Apple Silicon.

✨ New features and improvements

- [Learning rate schedules](https://thinc.ai/docs/api-schedules#schedule) can now take the training step as well as an arbitrary set of keyword arguments. This makes it possible to pass information such a the parameter name and last evaluation score to determine the learning rate (804).
- Added the [`plateau.v1`](https://thinc.ai/docs/api-schedules#plateau) schedule (842). This schedule scales the learning rate if training was found to be stagnant for a given period.
- The functionality of [`thinc-apple-ops`](https://github.com/explosion/thinc-apple-ops) is integrated into Thinc (#927). Starting with this version of Thinc, it is not necessary anymore to install `thinc-apple-ops`.

🔴 Bug fixes

- Fix the use of thread-local storage ([917](https://github.com/explosion/thinc/commit/5c46b82a47781ebff91c953c42c2361a3cdd4f15)).

⚠️ Backwards incompatibilities

- Thinc v9.0.0 only support Python 3.9 and later.
- Schedules are not generators anymore, but implementations of the [`Schedule`](https://thinc.ai/docs/api-schedules#schedule) class (804).
- `thinc.backends.linalg` has been removed (742). The same functionality is provided by implementations in BLAS that are better tested and more performant.
- `thinc.extra.search` has been removed (743). The beam search functionality in this module was strongly coupled to the spaCy transition parser and has therefore [moved to spaCy](https://github.com/explosion/spaCy/pull/11317) in v4.

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, kadarakos, shadeMe, svlandeg

🔴 Bug fixes

- Make [strings2arrays](https://thinc.ai/docs/api-layers#strings2arrays) work again for sequences of inequal length (918).
- Fix `cupy.cublas` import (921).

👥 Contributors

danieldk, honnibal, ines, svlandeg

✨ New features and improvements

Add the [ParametricAttention_v2](https://thinc.ai/docs/api-layers#parametricattention_v2) layer, which adds support for key transformations (913).

👥 Contributors

danieldk, honnibal, ines, svlandeg

✨ New features and improvements

Updates and binary wheels for Python 3.12.

👥 Contributors

adrianeboyd, honnibal, ines, svlandeg

✨ New features and improvements

To improve loading times and reduce conflicts, MXNet and TensorFlow are no longer imported automatically (890).

⚠️ Backwards incompatibilities

MXNet and TensorFlow support needs to be enabled explicitly. Previously, MXNet and TensorFlow were imported automatically if they were available in the current environment.

To enable MXNet:

python
from thinc.api import enable_mxnet
enable_mxnet()


To enable TensorFlow:

python
from thinc.api import enable_tensorflow
enable_tensorflow()


With spaCy CLI commands you can provide this custom code using `-c code.py`. For training use `spacy train -c code.py` and to package your code with your pipeline use `spacy package -c code.py`.

Future deprecation warning: built-in MXNet and TensorFlow support will be removed in Thinc v9. If you need MXNet or TensorFlow support in the future, you can transition to using a custom copy of the current `MXNetWrapper` or `TensorFlowWrapper` in your package or project.

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, svlandeg

🔴 Bug fixes

- Support zero-length batches and hidden sizes in `reduce_{max,mean,sum}` (882).
- Preserve values with dtype for `NumpyOps/CupyOps.asarray` (897).

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, svlandeg