Thinc

✨ New features and improvements

- Extend support for `typing_extensions` up to v4.1.x (for Python 3.7 and earlier).
- Various fixes in the test suite.

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, shadeMe

✨ New features and improvements

- Make [`Ops.asarray`](https://thinc.ai/docs/api-backends#asarray) implementations more robust.

🔴 Bug fixes

- Fix issue 624: Support CPU inference for models trained with gradient scaling.
- Fix issue 633: Fix invalid indexing in `Beam` when no states have valid transitions.
- Fix issue 639: Improve PyTorch `Tensor` handling in `CupyOps.asarray`.
- Fix issue 649: Clamp inputs in `Ops.sigmoid` to prevent overflow.
- Fix issue 651: Fix type safety issue with model ID assignment.
- Fix issue 653: Correctly handle Tensorflow GPU tensors in tests.
- Fix issue 660: Make `is_torch_array` work without PyTorch installed.
- Fix issue 664: Fix out of-bounds writes in `CupyOps.adam` and `NumpyOps.adam`.

⚠️ Backwards incompatibilities

- The `init` implementations for layers no longer return [`Model`](https://thinc.ai/docs/api-model#model).

📖 Documentation and examples

- Add [notebook demonstrating Bloom embeddings](https://github.com/explosion/thinc/blob/master/examples/bloom_embeddings.ipynb).
- Fix LSTM benchmark example.
- Update installation instructions.

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, kadarakos, koaning, notplus, richardpaulhudson, shadeMe

🔴 Bug fixes

- Fix issue 610: Improve compatibility with PyTorch versions before v1.9.0.

👥 Contributors

adrianeboyd, danieldk

✨ New features and improvements

- Add new activation functions: [`ClippedLinear.v1`](https://thinc.ai/docs/api-layers#clippedlinear), [`Gelu.v1`](https://thinc.ai/docs/api-layers#gelu), [`HardSigmoid.v1`](https://thinc.ai/docs/api-layers#hardsigmoid), [`HardSwish.v1`](https://thinc.ai/docs/api-layers#hardswish), [`HardSwishMobilenet.v1`](https://thinc.ai/docs/api-layers#hardswishmobilenet), [`HardTanh.v1`](https://thinc.ai/docs/api-layers#hardtanh), [`ReluK.v1`](https://thinc.ai/docs/api-layers#reluk), and [`Swish.v1`](https://thinc.ai/docs/api-layers#swish).
- Automatically set the GPU allocator to PyTorch when PyTorch models are loaded through `PyTorchWrapper` on GPU to avoid [memory contention between CuPy and PyTorch](https://thinc.ai/docs/usage-frameworks#memory-contention).
- Support big endian platforms through [`thinc-bigendian-ops`](https://github.com/andrewsi-z/thinc-bigendian-ops) and consistently serialize model data with little endian byte order.
- Add [`Softmax.v2`](https://thinc.ai/docs/api-layers#softmax) with support for softmax with temperature and optional normalization.
- Add [`CategoricalCrossentropy.v3`](https://thinc.ai/docs/api-loss#categorical_crossentropy) and [`SequenceCategoricalCrossentropy.v3`](https://thinc.ai/docs/api-loss#sequence_categorical_crossentropy) with support for label smoothing.
- Speed up [`CupyOps.maxout`](https://thinc.ai/docs/api-backends#maxout) by [exploiting GPU parallelism](https://github.com/explosion/thinc/pull/579#issue-1113932021) better.
- Support sequence lengths in the `NumpyOps.seq2col` and `CupyOps.seq2col` implementations of [`Ops.seq2col`](https://thinc.ai/docs/api-backends#seq2col) to determine padding.
- [Improve performance](https://github.com/explosion/thinc/pull/585#issue-1124058029) of [`Ragged`](https://thinc.ai/docs/api-types#ragged).
- Support [`Ragged`](https://thinc.ai/docs/api-types#ragged) arrays in [`expand_window.v1`](https://thinc.ai/docs/api-layers#expand_window).

🔴 Bug fixes

- Fix issue 552: Do not backpropagate `Inf`/`NaN` out of PyTorch layers when using mixed-precision training.
- Fix issue 578: Correctly cast the threshold argument of `CupyOps.mish` and correct an equation in `Ops.backprop_mish`.
- Fix issue 587: Correct invariant checks in `CategoricalCrossentropy.get_grad`.
- Fix issue 592: Update `murmurhash`requirement.
- Fix issue 594: Do not sort positional arguments in `Config`.

⚠️ Backwards incompatibilities

- The `out` keyword argument of `Ops.mish` and `Ops.backprop_mish` is replaced by `inplace` for consistency with other activations.

📖Documentation and examples

- Update [example Jupyter notebooks](https://github.com/explosion/thinc/#-selected-examples-and-notebooks) for the current Thinc version.

👥 Contributors

adrianeboyd, andrewsi-z, danieldk, honnibal, ines, Jette16, kadarakos, kianmeng, polm, svlandeg, thatbudakguy

🔴 Bug fixes

- Fix issue 553: Switch torch tensor type with `set_ops` and `use_ops`.
- Fix issue 554: Always restore original ops after `use_ops`.

👥 Contributors

adrianeboyd, danieldk, ryndaniels, svlandeg

✨ New features and improvements

- Speed up GPU training time with up to ~25% by using cuBLAS for computing Frobenius norms in gradient clipping.
- Give preference to [`AppleOps`](https://github.com/explosion/thinc-apple-ops) (if available) when calling `get_ops("cpu")`.
- Support missing values in `CategoricalCrossEntropy` when the labels are integers.
- Provide the option to run [`model.walk`](https://thinc.ai/docs/api-model#walk) with depth-first traversal.
- Wrap `forward`/`init` callbacks of a `Model` in `with_debug` and `with_nvtx_range` to facilitate recursively instrumenting models.

🔴 Bug fixes

- Fix issue 537: Fix `replace_node` on nodes with indirect node refs.

👥 Contributors

adrianeboyd, danieldk, honnibal, ines, svlandeg