Coremltools

Whats New

* New **documentation** available at [http://coremltools.readme.io](http://coremltools.readme.io/).
* New converter path to directly convert **PyTorch** models without going through ONNX.
* Enhanced **TensorFlow 2** conversion support, which now includes support for dynamic control flow and LSTM layers. Support for several popular models and architectures, including Transformers such as GPT and BERT-variants.
* New **unified conversion API** `ct.convert()` for converting PyTorch and TensorFlow (including `tf.keras`) models.
* New **Model Intermediate Language (MIL)** builder library to either build neural network models directly or [implement composite operations](https://coremltools.readme.io/docs/composite-operators).
* New utilities to configure inputs while converting from PyTorch and TensorFlow, using `ct.convert()` with `ct.ImageType()`, `ct.ClassifierConfig()`, etc., see details: https://coremltools.readme.io/docs/neural-network-conversion.
* [onnx-coreml](https://github.com/onnx/onnx-coreml) converter is now moved under coremltools and can be accessed as `ct.converters.onnx.convert()`.

Deprecations

* Deprecated the following methods
* `NeuralNetworkShaper` class.
* `get_allowed_shape_ranges()`.
* `can_allow_multiple_input_shapes()`.
* `visualize_spec()` method of the `MLModel` class.
* `quantize_spec_weights()`, instead use the `quantize_weights()` method.
* `get_custom_layer_names()`,` replace_custom_layer_name()`, `has_custom_layer()`, moved them to internal methods.

* Added deprecation warnings for, will be deprecated in next major release.
* `convert_neural_network_weights_to_fp16()`, `convert_neural_network_spec_weights_to_fp16()`. Instead use the `quantize_weights()` method. See https://coremltools.readme.io/docs/quantization for details.

Known Issues

* Tensorflow 2 model conversion is supported for models with 1 concrete function.
* Conversion for TensorFlow and PyTorch models with quantized weights is currently not supported.
* `coremltools.utils.rename_feature` does not work correctly in renaming the output feature of a model of type neural network classifier
* `leaky_relu` layer is not added yet to the PyTorch converter, although its supported in MIL and the Tensorflow converters.

- Added support for `tf.einsum` op
- Bug fixes in image pre-processing error handling, quantization function for the `embeddingND` layer, conversion of `tf.stack` op
- Updated the transpose removal mlmodel pass
- Fixed import statement to support scikit-learn >=0.21 (sapieneptus )
- Added deprecation warnings for class `NeuralNetworkShaper` and methods `visualize_spec`, `quantize_spec_weights`
- Updated the names of a few functions that were unintentionally exposed to the public API, to internal, by prepending with underscore. The original methods still work but deprecation warnings have been added.

Release Notes

Bug Fixes

* Add support for converting Softplus layer in coremltools.
* Fix in gelu and layer norm fusion pass.
* Simplified build & CI setup.
* Fixed critical numpy

This release includes new op conversion supports, bug fixes, and improved graph optimization passes.

Install/upgrade to the latest `coremltools` with `pip install --upgrade coremltools`.

More details can be found in [neural-network-guide.md](https://github.com/apple/coremltools/blob/master/docs/NeuralNetworkGuide.md).

Changes:

- Add support for TensorFlow 2.x file format (.h5, SavedModel, and concrete functions).
- Add support for several new ops, such as `AddV2`, `FusedBatchNormV3`.
- Bug fixes in the Tensorflow converter's op fusion graph pass.

Known Issues:

- `tf.keras` model conversion supported only with TensorFlow 2
- Currently, there are issues while invoking the TensorFlow 2.x model conversion in Python 2.x.
- Currently, there are issues while converting `tf.keras` graphs that contain recurrent layers.

We are happy to announce the official release of coremltools 3 which aligns with Core ML 3. It includes a new version of the .mlmodel specification (version 4) which brings with it support for:

* Updatable models - Neural Network and KNN
* More dynamic and expressive neural networks - approx. 100 more layers added compared to Core ML 2
* Dynamic control flows
* Nearest neighbor classifiers
* Recommenders
* Linked models
* Sound analysis preprocessing
* Runtime adjustable parameters for on-device update

This version of coremltools also includes a new converter path for TensorFlow models. The [tfcoreml converter](https://github.com/tf-coreml/tf-coreml) has been updated to include this new path to convert to specification 4 which can handle control flow and cyclic tensor flow graphs.

Control flow example can be found [here](https://github.com/apple/coremltools/blob/master/examples/Neural_network_control_flow_power_iteration.ipynb).

Updatable Models

Core ML 3 supports an on-device update of models. Version 4 of the `.mlmodel` specification can encapsulate all the necessary parameters for a model update. Nearest neighbor, neural networks and pipeline models can all be made updatable.
Updatable neural networks support the training of convolution and fully connected layer weights (with back-propagation through many other layers types). Categorical cross-entropy and mean squared error losses are available along with stochastic gradient descent and Adam optimizers.
See examples of how to convert and create [updatable models](https://github.com/apple/coremltools/tree/master/examples/updatable_models).
See the [MLUpdateTask API reference](https://developer.apple.com/documentation/coreml/mlupdatetask) for how to update a model from within an app.

Neural Networks

* Support for new layers in Core ML 3 added to the `NeuralNetworkBuilder`
* Exact rank mapping of multi dimensional array inputs
* Control Flow related layers (branch, loop, range, etc.)
* Element-wise unary layers (ceil, floor, sin, cos, gelu, etc.)
* Element-wise binary layers with broadcasting (addBroadcastable, multiplyBroadcastable, etc)
* Tensor manipulation layers (gather, scatter, tile, reverse, etc.)
* Shape manipulation layers (squeeze, expandDims, getShape, etc.)
* Tensor creation layers (fillDynamic, randomNormal, etc.)
* Reduction layers (reduceMean, reduceMax, etc.)
* Masking / Selection Layers (whereNonZero, lowerTriangular, etc.)
* Normalization layers (layerNormalization)
* For a full list of supported layers in Core ML 3, check out Core ML [specification documentation](https://apple.github.io/coremltools/coremlspecification/sections/NeuralNetwork.html) or [NeuralNetwork.proto](https://github.com/apple/coremltools/blob/c6e7d15e3aef676a60247fea235da58aedbfcfd7/mlmodel/format/NeuralNetwork.proto#L535).
* Support conversion of recurrent networks from [TensorFlow](https://github.com/tf-coreml/tf-coreml/releases)