PyTorch 2.0 stack support
We are very excited by the newly announced PyTorch 2.0 stack and you can try it using Accelerate on any model by using the `dynamo_backend` argument of the `Accelerator`, or when filling your config with `accelerate config`.
Note that to get the best performance, we recommend:
- using an Ampere GPU (or more recent)
- sticking to fixed shaped for now
* Add support for torch dynamo by sgugger in 829
New CLI commands
* Added two new commands, `accelerate config update` and `accelerate config default`. The first will update a config file to have the latest keys added from latter releases of Accelerate, and the second will create a default configuration file automatically mimicking `write_default_config()` introduced in 851 and 853 by muellerzr
* Also introduced a filterable help for `accelerate launch` which will show options relevant to the choices shown, such as `accelerate launch --multi_gpu` will show launch parameters relevant to multi-gpu training.
What's new?
* fix 🐛 by pacman100 in 836
* Deepspeed example should use gather_for_metrics by HammadB in 821
* Highlight selection with pretty colors by muellerzr in 839
* Add `join_uneven_inputs` context manager to Accelerator by Chris-hughes10 in 820
* Introduce `default-config` command by muellerzr in 840
* Fix log error and add log level to get_logger by muellerzr in 842
* Fix if/else by muellerzr in 849
* Fix complete_cv example by muellerzr in 848
* Refactor Accelerate config and introduce a multi-argument CLI interface by muellerzr in 851
* Clean up, add update command by muellerzr in 853
* Revert "Update pr docs actions by mishig25 in 827)"
* Switch default log to warn by muellerzr in 859
* Remove mixed precision hook as part of the unwrap_model by muellerzr in 860
* update deepspeed error message wrt `batch_size` by pacman100 in 861
* fix failing deepspeed test by pacman100 in 868
* Even more log level refined, leave alone if not explicitly set by muellerzr in 871
* Solve pickling issues by muellerzr in 872
* Spring cleaning by muellerzr in 865
* fixing lr_scheduler prepare issue when using pytorch nightly by pacman100 in 878
* fix fsdp state_dict_config because of PyTorch changes by pacman100 in 877
* Update deprecated logging warn by SHi-ON in 881
* fix a bug by xiaohu2015 in 887
* Allow safetensors offload by sgugger in 873
* fixing lr scheduler for pytorch nightly by pacman100 in 884
* Prefix all accelerate env vars with ACCELERATE by muellerzr in 890
* fix prefix issues in tests by pacman100 in 891
* Fix windows cli selector by muellerzr in 893
* Better description for improper kwargs by muellerzr in 894
* Support bfloat16 in load_offloaded_weight by sgugger in 892
Significant community contributions
The following contributors have made significant changes to the library over the last release:
* Chris-hughes10
* Add `join_uneven_inputs` context manager to Accelerator (820)