Keras

Areas of improvement

- Bug fixes.
- New APIs: sequence generation API `TimeseriesGenerator`, and new layer `DepthwiseConv2D`.
- Unit tests / CI improvements.
- Documentation improvements.

API changes

- Add new sequence generation API `keras.preprocessing.sequence.TimeseriesGenerator`.
- Add new convolutional layer `keras.layers.DepthwiseConv2D`.
- Allow weights from `keras.layers.CuDNNLSTM` to be loaded into a `keras.layers.LSTM` layer (e.g. for inference on CPU).
- Add `brightness_range` data augmentation argument in `keras.preprocessing.image.ImageDataGenerator`.
- Add `validation_split` API in `keras.preprocessing.image.ImageDataGenerator`. You can pass `validation_split` to the constructor (float), then select between training/validation subsets by passing the argument `subset='validation'` or `subset='training'` to methods `flow` and `flow_from_directory`.

Breaking changes

- As a side effect of a refactor of `ConvLSTM2D` to a modular implementation, recurrent dropout support in Theano has been dropped for this layer.

Credits

Thanks to our 28 contributors whose commits are featured in this release:

DomHudson, Dref360, VitamintK, abrad1212, ahundt, bojone, brainnoise, bzamecnik, caisq, cbensimon, davinnovation, farizrahman4u, fchollet, gabrieldemarmiesse, khosravipasha, ksindi, lenjoy, masstomato, mewwts, ozabluda, paulpister, sandpiturtle, saralajew, srjoglekar246, stefangeneralao, taehoonlee, tiangolo, treszkai

Areas of improvement

- Bug fixes
- Performance improvements
- Improvements to example scripts

API changes

- Allow for stateful metrics in `model.compile(..., metrics=[...])`. A stateful metric inherits from `Layer`, and implements `__call__` and `reset_states`.
- Support `constants` argument in `StackedRNNCells`.
- Enable some TensorBoard features in the `TensorBoard` callback (loss and metrics plotting) with non-TensorFlow backends.
- Add `reshape` argument in `model.load_weights()`, to optionally reshape weights being loaded to the size of the target weights in the model considered.
- Add `tif` to supported formats in `ImageDataGenerator`.
- Allow auto-GPU selection in `multi_gpu_model()` (set `gpus=None`).
- In `LearningRateScheduler` callback, the scheduling function now takes an argument: `lr`, the current learning rate.

Breaking changes

- In `ImageDataGenerator`, change default interpolation of image transforms from nearest to bilinear. This should probably not break any users, but it is a change of behavior.

Credits

Thanks to our 37 contributors whose commits are featured in this release:

DalilaSal, Dref360, GalaxyDream, GarrisonJ, Max-Pol, May4m, MiliasV, MrMYHuang, N-McA, Vijayabhaskar96, abrad1212, ahundt, angeloskath, bbabenko, bojone, brainnoise, bzamecnik, caisq, cclauss, dsadulla, fchollet, gabrieldemarmiesse, ghostplant, gorogoroyasu, icyblade, kapsl, kevinbache, mendesmiguel, mikesol, myutwo150, ozabluda, sadreamer, simra, taehoonlee, veniversum, yongtang, zhangwj618

Areas of improvement

- Performance improvements (esp. convnets with TensorFlow backend).
- Usability improvements.
- Docs & docstrings improvements.
- New models in the `applications` module.
- Bug fixes.

API changes

- `trainable` attribute in `BatchNormalization` now disables the updates of the batch statistics (i.e. if `trainable == False` the layer will now run 100% in inference mode).
- Add `amsgrad` argument in `Adam` optimizer.
- Add new applications: `NASNetMobile`, `NASNetLarge`, `DenseNet121`, `DenseNet169`, `DenseNet201`.
- Add `Softmax` layer (removing need to use a `Lambda` layer in order to specify the `axis` argument).
- Add `SeparableConv1D` layer.
- In `preprocessing.image.ImageDataGenerator`, allow `width_shift_range` and `height_shift_range` to take integer values (absolute number of pixels)
- Support `return_state` in `Bidirectional` applied to RNNs (`return_state` should be set on the child layer).
- The string values `"crossentropy"` and `"ce"` are now allowed in the `metrics` argument (in `model.compile()`), and are routed to either `categorical_crossentropy` or `binary_crossentropy` as needed.
- Allow `steps` argument in `predict_*` methods on the `Sequential` model.
- Add `oov_token` argument in `preprocessing.text.Tokenizer`.

Breaking changes

- In `preprocessing.image.ImageDataGenerator`, `shear_range` has been switched to use degrees rather than radians (for consistency). This should not actually break anything (neither training nor inference), but keep this change in mind in case you see any issues with regard to your image data augmentation process.


Credits

Thanks to our 45 contributors whose commits are featured in this release:

Dref360, OliPhilip, TimZaman, bbabenko, bdwyer2, berkatmaca, caisq, decrispell, dmaniry, fchollet, fgaim, gabrieldemarmiesse, gklambauer, hgaiser, hlnull, icyblade, jgrnt, kashif, kouml, lutzroeder, m-mohsen, mab4058, manashty, masstomato, mihirparadkar, myutwo150, nickbabcock, novotnj3, obsproth, ozabluda, philferriere, piperchester, pstjohn, roatienza, souptc, spiros, srs70187, sumitgouthaman, taehoonlee, tigerneil, titu1994, tobycheese, vitaly-krumins, yang-zhang, ziky90

Areas of improvement

- Bug fixes and performance improvements.
- API improvements in Keras applications, generator methods.

API changes

- Make `preprocess_input` in all Keras applications compatible with both Numpy arrays and symbolic tensors (previously only supported Numpy arrays).
- Allow the `weights` argument in all Keras applications to accept the path to a custom weights file to load (previously only supported the built-in `imagenet` weights file).
- `steps_per_epoch` behavior change in generator training/evaluation methods:
- If specified, the specified value will be used (previously, in the case of generator of type `Sequence`, the specified value was overridden by the `Sequence` length)
- If unspecified and if the generator passed is a `Sequence`, we set it to the `Sequence` length.
- Allow `workers=0` in generator training/evaluation methods (will run the generator in the main process, in a blocking way).
- Add `interpolation` argument in `ImageDataGenerator.flow_from_directory`, allowing a custom interpolation method for image resizing.
- Allow `gpus` argument in `multi_gpu_model` to be a list of specific GPU ids.

Breaking changes

- The change in `steps_per_epoch` behavior (described above) may affect some users.

Credits

Thanks to our 26 contributors whose commits are featured in this release:

Alex1729, alsrgv, apisarek, asos-saul, athundt, cherryunix, dansbecker, datumbox, de-vri-es, drauh, evhub, fchollet, heath730, hgaiser, icyblade, jjallaire, knaveofdiamonds, lance6716, luoch, mjacquem1, myutwo150, ozabluda, raviksharma, rh314, yang-zhang, zach-nervana

This release amends release 2.1.0 to include a fix for an erroneous breaking change introduced in 8419.

This is a small release that fixes outstanding bugs that were reported since the previous release.

Areas of improvement

- Bug fixes (in particular, Keras no longer allocates devices at startup time with the TensorFlow backend. This was causing issues with Horovod.)
- Documentation and docstring improvements.
- Better CIFAR10 ResNet example script and improvements to example scripts code style.

API changes

- Add `go_backwards` to cuDNN RNNs (enables `Bidirectional` wrapper on cuDNN RNNs).
- Add ability to pass `fetches` to `K.Function()` with the TensorFlow backend.
- Add `steps_per_epoch` and `validation_steps` arguments in `Sequential.fit()` (to sync it with `Model.fit()`).

Breaking changes

None.

Credits

Thanks to our 14 contributors whose commits are featured in this release:

Dref360, LawnboyMax, anj-s, bzamecnik, datumbox, diogoff, farizrahman4u, fchollet, frexvahi, jjallaire, nsuh, ozabluda, roatienza, yakigac