Web Portal
* New look and feel
Neural Architecture Search
* **Breaking change**: ``nni.retiarii`` is no longer maintained and tested. Please migrate to ``nni.nas``.
+ Inherit ``nni.nas.nn.pytorch.ModelSpace``, rather than use ``model_wrapper``.
+ Use ``nni.choice``, rather than ``nni.nas.nn.pytorch.ValueChoice``.
+ Use ``nni.nas.experiment.NasExperiment`` and ``NasExperimentConfig``, rather than ``RetiariiExperiment``.
+ Use ``nni.nas.model_context``, rather than ``nni.nas.fixed_arch``.
+ Please refer to [quickstart](https://nni.readthedocs.io/en/v3.0pt1/tutorials/hello_nas.html) for more changes.
* A refreshed experience to construct model space.
+ Enhanced debuggability via ``freeze()`` and ``simplify()`` APIs.
+ Enhanced expressiveness with ``nni.choice``, ``nni.uniform``, ``nni.normal`` and etc.
+ Enhanced experience of customization with ``MutableModule``, ``ModelSpace`` and ``ParamterizedModule``.
+ Search space with constraints is now supported.
* Improved robustness and stability of strategies.
+ Supported search space types are now enriched for PolicyBaseRL, ENAS and Proxyless.
+ Each step of one-shot strategies can be executed alone: model mutation, evaluator mutation and training.
+ Most multi-trial strategies now supports specifying seed for reproducibility.
+ Performance of strategies have been verified on a set of benchmarks.
* Strategy/engine middleware.
+ Filtering, replicating, deduplicating or retrying models submitted by any strategy.
+ Merging or transforming models before executing (e.g., CGO).
+ Arbitrarily-long chains of middlewares.
* New execution engine.
+ Improved debuggability via SequentialExecutionEngine: trials can run in a single process and breakpoints are effective.
+ The old execution engine is now decomposed into execution engine and model format.
+ Enhanced extensibility of execution engines.
* NAS profiler and hardware-aware NAS.
+ New profilers profile a model space, and quickly compute a profiling result for a sampled architecture or a distribution of architectures (FlopsProfiler, NumParamsProfiler and NnMeterProfiler are officially supported).
+ Assemble profiler with arbitrary strategies, including both multi-trial and one-shot.
+ Profiler are extensible. Strategies can be assembled with arbitrary customized profilers.
Model Compression
* Compression framework is refactored, new framework import path is ``nni.contrib.compression``.
+ Configure keys are refactored, support more detailed compression configurations. [view doc](https://nni.readthedocs.io/en/v3.0pt1/compression/config_list.html)
+ Support multi compression methods fusion.
+ Support distillation as a basic compression component.
+ Support more compression targets, like ``input``, ``ouptut`` and any registered paramters.
+ Support compressing any module type by customizing module settings.
* Model compression support in DeepSpeed mode.
* Fix example bugs.
* Pruning
+ Pruner interfaces have fine-tuned for easy to use. [view doc](https://nni.readthedocs.io/en/v3.0pt1/reference/compression/pruner.html)
+ Support configuring ``granularity`` in pruners. [view doc](https://nni.readthedocs.io/en/v3.0pt1/compression/config_list.html#granularity)
+ Support different mask ways, multiply zero or add a large negative value.
+ Support manully setting dependency group and global group. [view doc](https://nni.readthedocs.io/en/v3.0/compression/config_list.html#global-group-id)
+ A new powerful pruning speedup is released, applicability and robustness have been greatly improved. [view doc](https://nni.readthedocs.io/en/v3.0pt1/reference/compression/pruning_speedup.html)
+ The end to end transformer compression tutorial has been updated, achieved more extreme compression performance. [view doc](https://nni.readthedocs.io/en/v3.0pt1/tutorials/new_pruning_bert_glue.html)
+ Fix config list in the examples.
* Quantization
+ Support using ``Evaluator`` to handle training/inferencing.
+ Support more module fusion combinations. [view doc](https://nni.readthedocs.io/en/v3.0pt1/compression/module_fusion.html)
+ Support configuring ``granularity`` in quantizers. [view doc](https://nni.readthedocs.io/en/v3.0pt1/compression/config_list.html#id6)
+ Bias correction is supported in the Post Training Quantization algorithm.
+ LSQ+ quantization algorithm is supported.
* Distillation
+ [DynamicLayerwiseDistiller](https://nni.readthedocs.io/en/v3.0pt1/reference/compression/distiller.html#dynamiclayerwisedistiller) and [Adaptive1dLayerwiseDistiller](https://nni.readthedocs.io/en/v3.0pt1/reference/compression/distiller.html#adaptive1dlayerwisedistiller) are supported.
* Compression documents now updated for the new framework, the old version please view [v2.10 doc](https://nni.readthedocs.io/en/v2.10/).
* New compression examples are under [`nni/examples/compression`](https://github.com/microsoft/nni/tree/v3.0pt1/examples/compression)
+ Create a evaluator: [`nni/examples/compression/evaluator`](https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/evaluator)
+ Pruning a model: [`nni/examples/compression/pruning`](https://github.com/microsoft/nni/tree/v3.0pt1/examples/compression/pruning)
+ Quantize a model: [`nni/examples/compression/quantization`](https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/quantization)
+ Fusion compression: [`nni/examples/compression/fusion`](https://github.com/microsoft/nni/tree/v3.0pt1/examples/compression/fusion)
Training Services
* **Breaking change**: NNI v3.0 cannot resume experiments created by NNI v2.x
* Local training service:
+ Reduced latency of creating trials
+ Fixed "GPU metric not found"
+ Fixed bugs about resuming trials
* Remote training service:
+ ``reuse_mode`` now defaults to ``False``; setting it to ``True`` will fallback to v2.x remote training service
+ Reduced latency of creating trials
+ Fixed "GPU metric not found"
+ Fixed bugs about resuming trials
+ Supported viewing trial logs on the web portal
+ Supported automatic recover after temporary server failure (network fluctuation, out of memory, etc)
* Get rid of IoC and remove unused training services.