Keras

Areas of improvement

- RNN improvements:
- Refactor RNN layers to rely on atomic RNN cells. This makes the creation of custom RNN very simple and user-friendly, via the `RNN` base class.
- Add ability to create new RNN cells by stacking a list of cells, allowing for efficient stacked RNNs.
- Add `CuDNNLSTM` and `CuDNNGRU` layers, backend by NVIDIA's cuDNN library for fast GPU training & inference.
- Add RNN Sequence-to-sequence example script.
- Add `constants` argument in `RNN`'s `call` method, making RNN attention easier to implement.
- Easier multi-GPU data parallelism via `keras.utils.multi_gpu_model`.
- Bug fixes & performance improvements (in particular, native support for NCHW data layout in TensorFlow).
- Documentation improvements and examples improvements.



API changes

- Add "fashion mnist" dataset as `keras.datasets.fashion_mnist.load_data()`
- Add `Minimum` merge layer as `keras.layers.Minimum` (class) and `keras.layers.minimum(inputs)` (function)
- Add `InceptionResNetV2` to `keras.applications`.
- Support `bool` variables in TensorFlow backend.
- Add `dilation` to `SeparableConv2D`.
- Add support for dynamic `noise_shape` in `Dropout`
- Add `keras.layers.RNN()` base class for batch-level RNNs (used to implement custom RNN layers from a cell class).
- Add `keras.layers.StackedRNNCells()` layer wrapper, used to stack a list of RNN cells into a single cell.
- Add `CuDNNLSTM` and `CuDNNGRU` layers.
- Deprecate `implementation=0` for RNN layers.
- The Keras progbar now reports time taken for each past epoch, and average time per step.
- Add option to specific resampling method in `keras.preprocessing.image.load_img()`.
- Add `keras.utils.multi_gpu_model` for easy multi-GPU data parallelism.
- Add `constants` argument in `RNN`'s `call` method, used to pass a list of constant tensors to the underlying RNN cell.

Breaking changes

- Implementation change in `keras.losses.cosine_proximity` results in a different (correct) scaling behavior.
- Implementation change for samplewise normalization in `ImageDataGenerator` results in a different normalization behavior.

Credits

Thanks to our 59 contributors whose commits are featured in this release!

Alok, Danielhiversen, Dref360, HelgeS, JakeBecker, MPiecuch, MartinXPN, RitwikGupta, TimZaman, adammenges, aeftimia, ahojnnes, akshaychawla, alanyee, aldenks, andhus, apbard, aronj, bangbangbear, bchu, bdwyer2, bzamecnik, cclauss, colllin, datumbox, deltheil, dhaval067, durana, ericwu09, facaiy, farizrahman4u, fchollet, flomlo, fran6co, grzesir, hgaiser, icyblade, jsaporta, julienr, jussihuotari, kashif, lucashu1, mangerlahn, myutwo150, nicolewhite, noahstier, nzw0301, olalonde, ozabluda, patrikerdes, podhrmic, qin, raelg, roatienza, shadiakiki1986, smgt, souptc, taehoonlee, y0z

The primary purpose of this release is to address an incompatibility between Keras 2.0.7 and the next version of TensorFlow (1.4). TensorFlow 1.4 isn't due until a while, but the sooner the PyPI release has the fix, the fewer people will be affected when upgrading to the next TensorFlow version when it gets released.

No API changes for this release. A few bug fixes.

Areas of improvement

- Bug fixes.
- Performance improvements.
- Documentation improvements.
- Better support for training models from data tensors in TensorFlow (e.g. Datasets, TFRecords). Add a related example script.
- Improve TensorBoard UX with better grouping of ops into name scopes.
- Improve test coverage.

API changes

- Add `clone_model` method, enabling to construct a new model, given an existing model to use as a template. Works even in a TensorFlow graph different from that of the original model.
- Add `target_tensors` argument in `compile`, enabling to use custom tensors or placeholders as model targets.
- Add `steps_per_epoch` argument in `fit`, enabling to train a model from data tensors in a way that is consistent with training from Numpy arrays.
- Similarly, add `steps` argument in `predict` and `evaluate`.
- Add `Subtract` merge layer, and associated layer function `subtract`.
- Add `weighted_metrics` argument in `compile` to specify metric functions meant to take into account `sample_weight` or `class_weight`.
- Make the `stop_gradients` backend function consistent across backends.
- Allow dynamic shapes in `repeat_elements` backend function.
- Enable stateful RNNs with CNTK.

Breaking changes

- The backend methods `categorical_crossentropy`, `sparse_categorical_crossentropy`, `binary_crossentropy` had the order of their positional arguments (`y_true`, `y_pred`) inverted. This change does not affect the `losses` API. This change was done to achieve API consistency between the `losses` API and the backend API.
- Move constraint management to be based on variable attributes. Remove the now-unused `constraints` attribute on layers and models (not expected to affect any user).

Credits

Thanks to our 47 contributors whose commits are featured in this release!

5ke, Alok, Danielhiversen, Dref360, NeilRon, abnera, acburigo, airalcorn2, angeloskath, athundt, brettkoonce, cclauss, denfromufa, enkait, erg, ericwu09, farizrahman4u, fchollet, georgwiese, ghisvail, gokceneraslan, hgaiser, inexxt, joeyearsley, jorgecarleitao, kennyjacob, keunwoochoi, krizp, lukedeo, milani, n17r4m, nicolewhite, nigeljyng, nyghtowl, nzw0301, rapatel0, souptc, srinivasreddy, staticfloat, taehoonlee, td2014, titu1994, tleeuwenburg, udibr, waleedka, wassname, yashk2810

Areas of improvement

- Improve generator methods (`predict_generator`, `fit_generator`, `evaluate_generator`) and add data enqueuing utilities.
- Bug fixes and performance improvements.
- New features: new `Conv3DTranspose` layer, new `MobileNet` application, self-normalizing networks.

API changes

- Self-normalizing networks: add `selu` activation function, `AlphaDropout` layer, `lecun_normal` initializer.
- Data enqueuing: add `Sequence`, `SequenceEnqueuer`, `GeneratorEnqueuer` to `utils`.
- Generator methods: rename arguments `pickle_safe` (replaced with `use_multiprocessing`) and `max_q_size ` (replaced with `max_queue_size`).
- Add `MobileNet` to the applications module.
- Add `Conv3DTranspose` layer.
- Allow custom print functions for model's `summary` method (argument `print_fn`).

- Add beta CNTK backend.
- TensorBoard improvements.
- Documentation improvements.
- Bug fixes and performance improvements.
- Improve style transfer example script.

API changes:

- Add `return_state` constructor argument to RNNs.
- Add `skip_compile` option to `load_model`.
- Add `categorical_hinge` loss function.
- Add `sparse_top_k_categorical_accuracy` metric.
- Add new options to `TensorBoard` callback.
- Add `TerminateOnNaN` callback.
- Generalize the `Embedding` layer to N (>=2) input dimensions.

- Documentation improvements.
- Docstring improvements.
- Update some examples scripts (in particular, new deep dream example).
- Bug fixes and performance improvements.

API changes:

- Add `logsumexp` and `identity` to backend.
- Add `logcosh` loss.
- New signature for `add_weight` in `Layer`.
- `get_initial_states` in `Recurrent` is now `get_initial_state`.