Mlagents

Major Changes
* TensorFlow 2.0.0 is now supported. This is done through the `tf.compat.v1` interface (https://github.com/Unity-Technologies/ml-agents/pull/2665)
* Vector observations from the Agent are now sent using the ISensor interface. (https://github.com/Unity-Technologies/ml-agents/pull/2813)
* Demonstration files recorded using previous versions are not compatible with 0.12.0
* Custom observations, custom actions, text observations, and text actions were removed. (https://github.com/Unity-Technologies/ml-agents/pull/2839)
* Raycast and spherecast observations are now done through a RayPerceptionSensor, and should be configured by adding a RayPerceptionSensorComponent3D to an Agent. See the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md) for more details. (https://github.com/Unity-Technologies/ml-agents/pull/2874)
* RayPerceptionSensor adds more configuration such as toggling between raycasts and spherecasts, and better debug display.
* RayPerception3D and RayPerception2D are deprecated and will be removed in the next release.

For instructions on how to migrate from 0.11.0 to 0.12.0 see the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md).

Minor Fixes and Improvements
* A bug in how RayPerception3D used the vertical offset parameters was fixed. This may cause a behavior change in your observations; see the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md) for more details. (https://github.com/Unity-Technologies/ml-agents/pull/2823)
* A bug which swapped the height and width dimensions in visual observation was fixed. (https://github.com/Unity-Technologies/ml-agents/pull/2919)
* Memories and previous actions are no longer passed between the environment and the trainer. These are now saved by the trainer and at inference. This affects the low-level API but existing models are still compatible. (https://github.com/Unity-Technologies/ml-agents/pull/2795, https://github.com/Unity-Technologies/ml-agents/pull/2913)
* Unity Inference Engine (Barracuda) was upgraded to 0.3.2, and it is now installed via the Unity Package Manager. Please see the [Barracuda release notes](https://github.com/Unity-Technologies/barracuda-release/blob/release/0.3.2/CHANGELOG.md) for the list of changes.
* An issue with setting num_batches_per_epoch past the number of batches in the buffer while using recurrent and BC was fixed (https://github.com/Unity-Technologies/ml-agents/pull/2965)
* Addressed an issue with setting pretraining steps to 0 (continuous pretraining) (https://github.com/Unity-Technologies/ml-agents/pull/2977)

Major Changes

* The BroadcastHub has been deprecated. If there is a training Python process, all LearningBrains in the scene will automatically be trained. If there is no Python process, inference will be used.
* The Brain ScriptableObjects have been removed. The Brain Parameters are now on the Agent and are referred to as Behavior Parameters. Make sure the Behavior Parameters is attached to the Agent GameObject.
* The setup for visual observations has changed significantly.
* Camera resolutions are no longer stored in the Brain Parameters.
* AgentParameters no longer stores lists of Cameras and RenderTextures
* To add visual observations to an Agent, you must now attach a CameraSensorComponent or RenderTextureComponent to the agent. The corresponding Camera or RenderTexture can be added to these in the editor, and the resolution and color/grayscale is configured on the component itself.
* The definition of the gRPC service has changed.
* The online BC training feature has been removed.

For instructions on how to migrate from 0.10.x to 0.11.0 see the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/0.11.0/docs/Migrating.md).

See the updated [Getting Started](https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Getting-Started-with-Balance-Ball.md#agent) and [Basic Guide](https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Basic-Guide.md#setting-up-the-environment-for-training) for more information.

Minor Fixes and Improvements

* Fixed an exception being thrown when dragging between two screens with different DPIs (2782)
* Improved hyperparameters for the GridWorld environment using SAC (2776)
* Fixed crash when SAC is used with Curiosity and Continuous Actions (2740)
* Sped up processing large vector observations (2717)
* Fixed bug causing excessive memory usage during inference (2722)
* Added support for multiple training areas in the GridWorld environment (2721)
* RayPerception.Perceive() was made abstract and RayPerception2D now correctly overrides it. (2788)

Fixes & Performance Improvements
- Fix crash when exporting a behavioral cloning model with LSTM (2679)
- Fix issue when using an SAC model with LSTM in the Unity Inference Engine (2698)
- Fixed bug with missing cameras in VisualPyramids and VisualPushBlock scenes (2672)
- Fixed an issue when batch size is a non-multiple of sequence length when using LSTM (2661)

New Features
- [Soft Actor-Critic](https://github.com/Unity-Technologies/ml-agents/blob/master/Training-SAC.md) (SAC) is added as a new trainer option, complementing Proximal Policy Optimization (PPO). SAC, an off-policy algorithm, is more sample-efficient (i.e., requires fewer environment steps). For environments that take a long time to execute a step (about >0.1 second or greater) this can lead to dramatic training speedups of around 3-5 times versus PPO. In addition to sample-efficiency, SAC has been shown to be robust to small variations in the environment and effective at exploring the environment to find optimal behaviors. See the [SAC documentation](https://github.com/Unity-Technologies/ml-agents/blob/master/Training-SAC.md) for more details.
- [Example environments](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md) have been updated to a new dark-theme visual style and colors have been standardized across all environments.
- Unity environment command line arguments can be passed through mlagents-learn. See [the documentation](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md) on how to use this feature.

Fixes and Improvements
- ML-Agents is now compatible with Python v3.7 and newer versions of Tensorflow up to 1.14.
- Fixed an issue when using recurrent networks and agents are destroyed. (2549)
- Fixed a memory leak during inference. (2541)
- The UnitySDK.log is no longer logged out, which fixes an issue with 2019 versions of the Unity Editor (2580).
- The Academy class no longer has a Done() method. All Done calls should be handled in the Agent (2519). See [Migrating](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md) for more information.
- C code was updated to follow Unity coding conventions.
- Fixed a crash that happens when enabling VAIL with a GAIL reward signal (2598)
- Other minor documentation enhancements and bug fixes.

Acknowledgements
- Thanks to tomatenbrei and everyone at Unity who contributed to v0.10.0.

Fixes and Improvements
- Fixed a bug in BCTrainer that would cause it to raise an exception when starting training (https://github.com/Unity-Technologies/ml-agents/pull/2505)
- Fixed a bug in the BananaIL scene that made it unplayable, due to missing reset parameters (https://github.com/Unity-Technologies/ml-agents/pull/2512)

New Features
- Added multi-gpu implementation for PPO. You can turn this on by using `--multi-gpu` option in [`mlagents-learn`](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-ML-Agents.md#command-line-training-options). In our own experiments we've observed a 38% boost on the update speed of Resnet using the [Obstacle Tower Environment](https://github.com/Unity-Technologies/obstacle-tower-env). (#2288)

Fixes and Improvements
- Improved the training performance by 20-30% when curiosity is used. We achieved this by updating the reward signal in parallel with the policy. (2362)
- Changed the Crawler example environment observation space to include relative observations. Increases performance in `CrawlerDynamic` scene. ([18f661](https://github.com/Unity-Technologies/ml-agents/commit/18f6616cdddd1b0dea14cabf169bed0e3245535f))
- Fixed a bug with online Behavior Cloning. (2384)
- Fixed a compile error on DemonstrationImporter.cs. ([b617de](https://github.com/Unity-Technologies/ml-agents/commit/b617de4401e3c9a362c6da70e5780f6c496faff0))

Acknowledgements

Thanks to everyone at Unity who contributed to v0.9.2, as well as: RunSwimFlyRich, niskander.