Mlagents

Major Changes
- `Agent.CollectObservations` now takes a VectorSensor argument. (3352, 3389)
- Added `Agent.CollectDiscreteActionMasks` virtual method with a `DiscreteActionMasker` argument to specify which discrete actions are unavailable to the Agent. (3525)
- We consolidated our API for `DiscreteActionMasker`. `SetMask` takes two arguments : the branch index and the list of masked actions for that branch. (3525)
- Beta support for ONNX export was added. If the `tf2onnx` python package is installed, models will be saved to `.onnx` as well as `.nn` format. Note that Barracuda 0.6.0 or later is required to import the `.onnx` files properly. (3101)
- Multi-GPU training and the `--multi-gpu` option has been removed temporarily. (3345)
- All Sensor related code has been moved to the namespace `MLAgents.Sensors`. (3519)
- All SideChannel related code has been moved to the namespace `MLAgents.SideChannels`. (3533)
- The following methods in the `Agent` class have been deprecated and will be removed in a later release (3557):
- `InitializeAgent()` was renamed to `Initialize()`
- `AgentAction()` was renamed to `OnActionReceived()`
- `AgentReset()` was renamed to `OnEpisodeBegin()`
- `Done()` was renamed to `EndEpisode()`
- `GiveModel()` was renamed to `SetModel()`

Minor Changes
- Monitor.cs was moved to Examples. (3372)
- Automatic stepping for Academy is now controlled from the AutomaticSteppingEnabled property. (3376)
- The GetEpisodeCount, GetStepCount, GetTotalStepCount and methods of Academy were changed to EpisodeCount, StepCount, TotalStepCount properties respectively. (3376)
- Several classes were changed from public to internal visibility. (3390)
- Academy.RegisterSideChannel and UnregisterSideChannel methods were added. (3391)
- A tutorial on adding custom SideChannels was added. (3391)
- The stepping logic for the Agent and the Academy has been simplified. (3448)
- Update Barracuda to 0.6.1-preview. (3584)
- The interface for `RayPerceptionSensor.PerceiveStatic()` was changed to take an input class and write to an output class, and the method was renamed to `Perceive()`. (3527)
- The checkpoint file suffix was changed from `.cptk` to `.ckpt`. (3470)
- The command-line argument used to determine the port that an environment will listen on was changed from `--port` to `--mlagents-port`. (3477)
- `DemonstrationRecorder` can now record observations outside of the editor. (3354)
- `DemonstrationRecorder` now has an optional path for the demonstrations. This will default to `Application.dataPath` if not set. (3354)
- `DemonstrationStore` was changed to accept a `Stream` for its constructor, and was renamed to `DemonstrationWriter`. (3354)
- The method `GetStepCount()` on the Agent class has been replaced with the property getter `StepCount`. (3476)
- `RayPerceptionSensorComponent` and related classes now display the debug gizmos whenever the Agent is selected (not just Play mode). (3484)
- Most fields on `RayPerceptionSensorComponent` can now be changed while the editor is in Play mode. The exceptions to this are fields that affect the number of observations. (3484)
- Most fields on `CameraSensorComponent` and `RenderTextureSensorComponent` were changed to private and replaced by properties with the same name. (3564)
- Unused static methods from the `Utilities` class (ShiftLeft, ReplaceRange, AddRangeNoAlloc, and GetSensorFloatObservationSize) were removed. (3495)
- The `Agent` class is no longer abstract. (3528)
- SensorBase was moved out of the package and into the Examples directory. (3528)
- `AgentInfo.actionMasks` has been renamed to `AgentInfo.discreteActionMasks`. (3539)
- `DecisionRequester` has been made internal (you can still use the DecisionRequesterComponent from the inspector). `RepeatAction` was renamed to`TakeActionsBetweenDecisions` for clarity. (3555)
- The `IFloatProperties` interface has been removed. (3570)
- Improved inference performance for models with multiple action branches. (3598)
- The interfaces to the `SideChannel` classes (on C and python) have changed to use new `IncomingMessage` and `OutgoingMessage` classes. These should make reading and writing data to the channel easier. (3596)

Bug Fixes and Improvements
- Various documentation changes and additions. (3603, 3574, 3577, 3568, 3567, 3551, 3582)
- Fixed logging issue with SAC. (3606)
- Fixed an issue when using GAIL with less than `batch_size` number of demonstrations. (3591)
- Made Unity lifecycle methods protected (`OnEnable`,` OnDisable`). (3590)
- SideChannel messages were lost if sent before the first call to reset from Python. (3573)
- Updated the ExpertPyramid.demo example demonstration file. (3613)
- Updated project version for example environments to 2018.4.18f1. (3618)
- Changed the `Product Name` in the example environments to remove spaces, so that the default build executable file doesn't contain spaces. (3612)

Bug Fixes
- Fixed an issue which caused self-play training sessions to consume a lot of memory. (3451)
- Fixed an IndexError when using GAIL or behavioral cloning with demonstrations recorded with 0.14.0 or later (3464)
- Updated the `gail_config.yaml` to work with per-Agent steps (3475)
- Fixed demonstration recording of experiences when the Agent is done. (3463)
- Fixed a bug with the rewards of multiple Agents in the gym interface (3471, 3496)

Major Changes

- A new self-play mechanism for training agents in adversarial scenarios was added (3194)
- Tennis and Soccer environments were refactored to enable training with self-play (3194, 3331)
- UnitySDK folder was split into a Unity Package (com.unity.ml-agents) and our examples were moved to the Project folder (3267)
- Academy is now a singleton and is no longer abstract (3210, 3184)
- In order to reduce the size of the API, several classes and methods were marked as internal or private.
- Some public fields on the Agent were trimmed (3342, 3353, 3269)
- Decision Period and on-demand decision checkboxes were removed from the Agent. on-demand decision is now the default (3243)
- Calling Done() on the Agent will reset it immediately and call the AgentReset virtual method (3291, 3242)
- The "Reset on Done" setting in AgentParameters was removed; this is now always true. AgentOnDone virtual method on the Agent was removed (3311, 3222)
- Trainer steps are now counted per-Agent, not per-environment as in previous versions. For instance, if you have 10 Agents in the scene, 20 environment steps now correspond to 200 steps as printed in the terminal and in Tensorboard (3113)

Minor Changes

- Barracuda was updated to 0.5.0-preview (3329)
- `--num-runs` option was removed from mlagents-learn (3155)
- Curriculum config files are now YAML formatted and all curricula for a training run are combined into a single file (3186)
- ML-Agents components, such as BehaviorParameters and various Sensor implementations, now appear in the Components menu (3231)
- Exceptions are now raised in Unity (in debug mode only) if NaN observations or rewards are passed (3221)
- RayPerception MonoBehavior, which was previously deprecated, was removed (3304)
- Uncompressed visual (i.e. 3d float arrays) observations are now supported. CameraSensorComponent and RenderTextureSensor now have an option to write uncompressed observations (3148)
- Agent’s handling of observations during training was improved so that an extra copy of the observations is - no longer maintained (3229)
- Error message for missing trainer config files was improved to include the absolute path (3230)
- Support for 2017.4 LTS was dropped (3121, 3168)
- Some documentation improvements were made (3296, 3292, 3295, 3281)

Bug Fixes
- Numpy warning when stats don’t exist (3251)
- A bug that caused RayPerceptionSensor to behave inconsistently with transforms that have non-1 scale was fixed (3321)
- Some small bugfixes to tensorflow_to_barracuda.py were backported from the barracuda release (3341)
- Base port in the jupyter notebook example was updated to use the same port that the editor uses (3283)

Minor Fixes and Improvements
* Fixed an issue where resuming training using `--load` would reset Tensorboard steps (3223)
* Fixed an issue where using Tensorflow on a GPU would consume all GPU memory (3219)

Major Changes
* The low level Python API has changed (3022). See the Low Level Python API documentation for more information.
* Parameters such as resolution and time scale are no longer configured in the Academy; these are now passed on the mlagents-learn command line. See the Migration Guide for more details. (2956)
* Offline Behavioral Cloning training was removed. To learn from demonstrations, use the GAIL and Behavioral Cloning features with either PPO or SAC (2969)
* Agents can now use sensors of child GameObjects as observations (3095)
* The RayPerceptionSensor now supports a layerMask option that is used in raycasting. (3111 )
* The official minimum version of Unity supported by ML-Agents is now 2018.4 LTS.

For instructions on how to migrate from 0.12.0 to 0.13.0 see the [Migration Guide](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Migrating.md).

Minor Fixes and Improvements
* A separation of statistics collection from the trainer with the StatsReporter and StatsWriter classes (3076, 3108)
* mlagents.envs was renamed to mlagents_envs
* A new AgentProcessor class and Trajectory abstraction is now live. Trainers now ingest trajectories which are assembled by the AgentProcessor. (3067)
* A bug that could cause the Academy to call a disabled agent’s _AgentReset() method is fixed (3072)
* Better error handling when the trainer configuration doesn’t contain a “default” entry (3063)
* Better error handling when there is a mismatch between the metacurriculum configuration and brains being trained (3034)
* A bug that prevented agents with different decision intervals from learning in the same scene is fixed (3181)

Fixes & Performance Improvements
* Fixed bug that could degrade training for recurrent networks (https://github.com/Unity-Technologies/ml-agents/pull/3066). If you are on 0.12.0 and using LSTMs to train your agents, we recommend you upgrade to 0.12.1 and retrain your model.
* Fixed bug in the environment setup in a Jupyter notebook (https://github.com/Unity-Technologies/ml-agents/pull/3068)
* Added extra logging on API versions in port numbers, to help debugging connection issues (https://github.com/Unity-Technologies/ml-agents/pull/3069)
* Minor documentation fixes (https://github.com/Unity-Technologies/ml-agents/pull/3070)