This is the release of TorchServe v0.6.0.
New Features
+ Support PyTorch 1.11 and Cuda 11.3 - Added [support](https://github.com/pytorch/serve/pull/1592) for PyTorch 1.11 and Cuda 11.3.
+ Universal Auto Benchmark and Dashboard Tool - Added [one command line tool](https://github.com/pytorch/serve/tree/master/benchmarks#Auto-Benchmarking-with-Apache-Bench) for model analyzer to get benchmark report([sample](https://github.com/pytorch/serve/blob/master/benchmarks/sample_report.md)) and dashboard on any device.
+ HuggingFace model parallelism integration - Added [example](https://github.com/pytorch/serve/pull/1510) for HuggingFace model parallelism integration.
Build and CI
+ Added nightly benchmark dashboard - Added [nightly benchmark dashboard](https://github.com/pytorch/serve/pull/1589).
+ Migrated CI, nightly binary and docker build to github workflow - Added [CI](https://github.com/pytorch/serve/pull/1576), [docker](https://github.com/pytorch/serve/pull/1562) migration.
+ Fixed gpu regression test `buildspec.yaml` - Added [fixing](https://github.com/pytorch/serve/pull/1479) for gpu regression test `buildspec.yaml`.
Documentation
+ Updated documentation - Updated [TorchServe](https://github.com/pytorch/serve/pull/1583), [benchmark](https://github.com/pytorch/serve/pull/1572), [snapshot](https://github.com/pytorch/serve/pull/1483) and [configuration](https://github.com/pytorch/serve/pull/1551) documentation; fixed broken [documentation build](https://github.com/pytorch/serve/pull/1570)
Deprecations
+ Deprecated old `benchmark/automated` [directory](https://github.com/pytorch/serve/pull/1594) in favor of new Github Action based workflow
Improvements
+ Fixed workflow threads cleanup - Added [fixing](https://github.com/pytorch/serve/issues/1511) to clean workflow inference threadpool.
+ Fixed empty model url - Added [fixing](https://github.com/pytorch/serve/pull/1523) for empty model url in model archiver.
+ Fixed load model failure - Added [support](https://github.com/pytorch/serve/pull/1508) for loading a model directory.
+ HuggingFace text generation example - Added [text generation example](https://github.com/pytorch/serve/pull/1473).
+ Updated metrics json and qlog format log - Added [support](https://github.com/pytorch/serve/pull/1491) for metrics json and qlog format log in log4j2.
+ Added cpu, gpu and memory usage - Added [cpu, gpu and memory usage](https://github.com/pytorch/serve/pull/1453) in `benchmark-ab.py` report.
+ Added exception for `torch < 1.8.1` - Added [exception](https://github.com/pytorch/serve/pull/1556) to notify `torch < 1.8.1`.
+ Replaced hard code in `install_dependencies.py` - Added [sys.executable](https://github.com/pytorch/serve/pull/1555) in `install_dependencies.py`.
+ Added default envelope for workflow - Added [default envelope](https://github.com/pytorch/serve/pull/1550) in model manager for workflow.
+ Fixed multiple docker build errors - Fixed [/home/venv write permission](https://github.com/pytorch/serve/pull/1514), [typo](https://github.com/pytorch/serve/pull/1561) in docker and added [common requirements](https://github.com/pytorch/serve/pull/1509) in docker.
+ Fixed snapshot test - Added [fixing](https://github.com/pytorch/serve/pull/1524) for snapshot test.
+ Updated `model_zoo.md` - Added [dog breed, mmf and BERT](https://github.com/pytorch/serve/pull/1497) in model zoo.
+ Added `nvgpu` in common requirements - Added [nvgpu](https://github.com/pytorch/serve/pull/1474) in common dependencies.
+ Fixed Inference API ping response - Fixed [typo](https://github.com/pytorch/serve/pull/1541) in Inference API ping response.
Platform Support
Ubuntu 16.04, Ubuntu 18.04, MacOS 10.14+, Windows 10 Pro, Windows Server 2019, Windows subsystem for Linux (Windows Server 2019, WSLv1, Ubuntu 18.0.4). TorchServe now requires Python 3.8 and above.
GPU Support