=========================================
This release contains a lot of bug fixes and improvements + new features, to prepare the upcoming release candidate.
Highlights:
- Many computation and compilation speed up
- More numerical stability by default for some graph
- Jenkins (gpu tests run on PR in addition to daily buildbot)
- Better handling of corner cases for theano functions and graph optimizations
- More graph optimization (faster execution and smaller graph, so more readable)
- Less c code compilation
- Better Python 3.5 support
- Better numpy 1.12 support
- Support newer Mac and Windows version
- Conda packages for Mac, Linux and Windows
- Theano scripts now works on Windows
- scan with checkpoint (trade off between speed and memory usage, useful for long sequences)
- Added a bool dtype
- New GPU back-end:
- float16 storage
- better mapping between theano device number and nvidia-smi number, using the PCI bus ID of graphic cards
- More pooling support on GPU when cuDNN isn't there
- ignore_border=False is now implemented for pooling
A total of 111 people contributed to this release since 0.8.0, see the list at the bottom.
Interface changes:
- New pooling interface
- Pooling parameters can change at run time
- When converting empty list/tuple, now we use floatX dtype
- The MRG random generator now try to infer the broadcast pattern of its output
- Move softsign out of sandbox to theano.tensor.nnet.softsign
- Roll make the shift be modulo the size of the axis we roll on
- Merge CumsumOp/CumprodOp into CumOp
- round() default to the same as NumPy: half_to_even
Convolution updates:
- Multi-cores convolution and pooling on CPU
- New abstract 3d convolution interface similar to the 2d convolution interface
- Dilated convolution
GPU:
- cuDNN: support versoin 5.1 and wrap batch normalization (2d and 3d) and RNN functions
- Multiple-GPU, synchrone update (via platoon, use NCCL)
- GpuAdvancedSubtensor in new back-end
- Gemv(matrix-vector product) speed up for special shape
- Support for MaxAndArgMax for some axis combination
- Support for solve (using cusolver), erfinv and erfcinv
- cublas gemv workaround when we reduce on an axis with a dimensions size of 0
- Warn user that some cuDNN algorithms may produce unexpected results in certain environments
for convolution backward filter operations
New features:
- Add gradient of solve, tensorinv (CPU), tensorsolve (CPU) searchsorted (CPU)
- Add Multinomial Without Replacement
- conv3d2d support full and half mode (REMOVE?)
- Add DownsampleFactorMaxGradGrad.grad
- Allow partial evaluation of compiled function
- More Rop support
- Indexing support ellipsis: a[..., 3], a[1,...,3]
- Added theano.tensor.{tensor5,dtensor5, ...}
- compiledir_format support device
- Added new Theano flag cmodule.age_thresh_use
Others:
- Speed up argmax only on gpu (without also needing the max)
- A few unfrequent bugfix
- More stack trace in error message
- Speed up cholesky grad
- log(sum(exp(...))) now get stability optimized
Other more detailed changes:
- Allow more then one output to be an destructive inplace
- Add flag profiling.ignore_first_call, useful to profile the new gpu back-end
- Doc/error message fixes/updates
- More support of negative axis
- Added the keepdims parameter to the norm function
- Crash fixes
- Make scan gradient more deterministic
- Add support for space in path on Windows
- remove ProfileMode (use Theano flag profile=True instead)