Torchserve

This is a hotfix release of TorchServe v0.4.2.

Improvements
+ **Fixed [the issue of port sharing between management and inference API](https://github.com/pytorch/serve/pull/1175)**
+ **Fixed [the issue of cleaning up tmp dir in model archiver](https://github.com/pytorch/serve/pull/1016)**
+ **Fixed [KFServing dockerfile](https://github.com/pytorch/serve/pull/1169)**

This is the release of TorchServe v0.4.1.

New Features

This is the release of TorchServe v0.4.0.

New Features
+ **Workflow support** - Added [support](https://github.com/pytorch/serve/tree/release_0.4.0/examples/Workflows) for sequential and parallel ensemble models with Language Translation and Computer Vision classification examples.
+ **S3 Model Store SSE support** - Added [support](https://github.com/pytorch/serve/blob/release_0.4.0/docs/management_api.md#register-a-model) for S3 server side model encryption via KMS.
+ **MMF-activity-recognition model example** - Added example [MMF-activity-recognition model](https://github.com/pytorch/serve/tree/master/examples/MMF-activity-recognition)

Patch release. Fixes Model Archiver to Recursively copy all artifacts

* Make --serialized-file an Optional Argument 994
* Recursively copy all files during archive 814

Highlights:
+ **Native windows support** - Added [support](https://github.com/pytorch/serve/blob/master/docs/torchserve_on_win_native.md) for TorchServe on Windows 10 pro and Windows Server 2019
+ **KFServing Integration** - Added support for v1 KFServing predict and explain [APIs](https://github.com/pytorch/serve/tree/master/kubernetes/kfserving) with auto-scaling and canary deployments for serving models in Kubeflow/KFServing
+ **MLFlow-TorchServe:** New [MLflow TorchServe deployment plugin](https://github.com/mlflow/mlflow-torchserve) for serving models for MLflow MLOps lifecycle
+ **Captum explanations** - Added explain API for [Captum](https://captum.ai/) model interpretability of different models
+ **AKS Support** - Added support for TorchServe deployment on [Azure Kubernetes Service](https://github.com/pytorch/serve/tree/master/kubernetes/AKS)
+ **GKE Support** - Added support for TorchServe deployment on [Google Kubernetes Service](https://github.com/pytorch/serve/tree/master/kubernetes/GKE)
+ **gRPC support** - Added support for gRPC based management and inference [APIs](https://github.com/pytorch/serve/blob/master/docs/grpc_api.md)
+ **Request Envelopes** - Added support for [request envelopes](https://github.com/pytorch/serve/blob/master/docs/request_envelopes.md) which parses request from multiple Model serving frameworks like Seldon, KFServing, without any modifications in the handler code

Highlights:
+ **Kubernetes Support** - Torchserve deployment in Kubernetes using [Helm Charts](https://helm.sh/) and a [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
+ **Prometheus metrics** - Added Prometheus as the default metrics framework
+ **Requirements.txt support​** - Added support to specify model specific dependencies as a requirements file within a mar archive; Cleanup of unused parameters and addition of relevant ones for torch-model-archiver
+ **Pytorch Scripted Models Support** - Scripted model versions added to model zoo; Added testing for scripted models
+ **Default Handler Refactor: (breaking changes)** The default handlers have been refactored for code reuse and enhanced post-processing support. More details in _Backwards Incompatible Changes_ section below
+ **Windows Support** - Added support for torchserve on windows subsystem for Linux
+ **AWS Cloud Formation Support** - Added support for multi-node [AutoScaling Group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/AutoScalingGroup.html) deployment, behind an [Elastic Load Balancer](https://aws.amazon.com/elasticloadbalancing/) using [Elastic File System](https://aws.amazon.com/efs/) as the backing store
+ **Benchmark and Testing Enhancements** - Added models in benchmark and sanity tests, support for throughput with batch processing in benchmarking, support docker for jmeter and apache benchmark tests
+ **Regression Suite Enhancements** - Added new POSTMAN based test cases for API and pytest based intrusive test cases
+ **Docker Improvements** - Consolidated dev and codebuild dockerfiles
+ **Install and Build Script Streamlining** - Unified install scripts, added code coverage and sanity script
+ **Python Linting** - More exhaustive python linting checks across Torchserve and Model Archiver

Backwards Incompatible Changes
+ **Default Handler Refactor**:
* The default handlers have been refactored for code reuse and enhanced post-processing support. The output format for some of the following examples/models has been enhanced to include additional details like score/class probability.
* [object detector](https://github.com/pytorch/serve/tree/issue_411/examples/object_detector/fast-rcnn)
* [image segmentor](https://github.com/pytorch/serve/tree/issue_411/examples/image_segmenter)
* The following default-handlers have been equipped with batch support. Due to batch support, [resnet_152_batch](https://github.com/pytorch/serve/tree/issue_411/examples/image_classifier/resnet_152_batch) example is not a custom handler example anymore.
* image_classifier
* object_detector
* image_segmenter
* The [index_to_name.json](https://github.com/pytorch/serve/blob/issue_411/docs/default_handlers.md#index_to_namejson) file use for the class to name mapping has been standardized across vision/text related default handlers
* Refactoring and code reuse have resulted into reduced boilerplate code in all the `serve/examples`.
* [Custom handler](https://github.com/pytorch/serve/blob/issue_411/docs/custom_service.md) documentation has been restructured and enhanced to facilitate the different possible ways to build simple or complex custom handlers