Fastestimator

This is a new major release of FastEstimator. Here are the highlights of new release:

Backend:
* New framework backend: TF 2.11, torch 2.0.1
* Introduced `IfElse` Helper function to help create simplified code

Apphub:
* New Apphub: keypoint detection - HRNet, 3D segmentation - 3DUnet+, Line Search, LeViT
* Added use your own dataset section to help with adapting to new task
* Updated SimCLR apphub for better efficiency
* Fix in-place operation in PGGAN in pytorch that cause graph to fail randomly

Dataset:
* New dataset class - Interleave dataset to switch dataset on per-step basis for multi-task learning.
* Added keypoint to mscoco dataset
* Added Pascal VOC dataset, MedMnist dataset
* Add filtering functionality to the csv dataset class
* Created custom pycocotools API to resolve pycocotool compilation issue
* Migrated several dataset hosting to google drive for stability
* Improved stability of batchdataset probability sampling

Pipeline:
* Fixed eval logging for multi-ds users
* Pipeline can now be instantiated without datasets

NumpyOp:
* Fixed issue encountered with Onehot encoding
* Added probability to OneOf NumpyOp

CLI:
* enabled warmup, eager, summary argument in run cli

Network:
* Fixed an issue when model's optimizer is None, the model_lr will not be printed at the end of training
* Fixed a performance issue of torch unet
* New patch-based inferencing class - Slicer

TensorOp:
* enabled multi-dimensional support for cross entropy loss
* Since Mixup and Cutmix no longer uses MixLoss, remove the class and update docstring
* Update focal loss's default mode to be consistent with other lossOp
* Resolved L1 loss dimension mismatch issue
* Focal loss rework
* Introduced RepeatOp for tensor operations
* Added probability to OneOf TensorOp

Visualization:
* Add BatchDisplay and GridDisplay traces
* FE logging visualization will work with single file
* Extending visualization to keypoints and masks

Trace:
* CSV logger rework
* added classification AUC trace
* Dice Trace rework
* fixed an issue of restore with unhashable param loading

Traceability:
* Updated pytorch model summary for traceability report
* Fixed multi-gpu model traceability graph
* Traceability Report now displays hardware information

Others:
* Updated yapf setting to work with recent yapf versions
* new benchmarking tool for better speed and resource monitoring
* New support matrix added to help user install past FE versions
* Fix ipython version due to its recent release upgrade that no longer supports python 3.8 below
* Updated Mac installation guide

Thank everyone who provided their feedbacks and made contribution to FE.

This release features several dependency bug fixes which caused issue during installation. Specifically, here is a list of notable changes in dependencies:
* numpy: specified a maximum numpy version due to numpy stopped supporting np.bool in recent versions.
* scikit-learn: changed sklearn to scikit-learn due to their recent name change and occasional downtime in the installation
* uncertainty-calibration: upgraded to a more recent version due to their dependency of sklearn

Moreover, this release will increase the stability of the package by resolving all open-ended dependency list. This will avoid future issues installing this release.

Finally, this release also incorporates a critical bug fix: TensorOp Oneof. TensorOp version of Oneof works properly now.

Release Note

Backend:
* Upgraded tensorFlow backend to 2.9.1
* upgraded PyTorch backend to 1.10.2
* Both backends now require CUDA > 11

NumpyOp:
* Re-designed `ReadMat` Op so that it can read arbiturary keys in .mat files and create different keys.

TensorOp:
* `Resize3D` can achieve data resizing while preserving the gradient for 3D data
* Added a new `Dice` Op that can be used in Loss and Trace. It supports a wide range of formulations. [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/dice_loss/unet_torch.py#L103-L107)]
* Added a new L1 Loss Op that supports L1 loss and its variants (Smoothed L1, Huber Loss) [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/l1_loss/retinanet.py#L567-L570)]
* Added support of focal loss for classification and segmentation problems [[code snippet](https://github.com/fastestimator-util/fastestimator-misc/blob/22602865abd6a1c9ac099a0aeaa2eedc1dde66f1/examples/1.5/retinanet/retinanet_tf.py#L576-L578)]

Search API:
* Redesigned Search API for general purpose hyper-parameter tuning or variety robustness testing

Pipeline:
* Reduced the key pruning warning frequency

Network:
* improved training speed on multi-gpu
* supported modifying layer's retrainable property through trace for fine-tuning purpose
* supported PyTorch training with partial gradient available
* Supported TensorFlow sub-classing model definition

Estimator:
* Added Evaluation log log during eval phase to show eval progress

Visualization
* Switched the visualization engine to plotly, now FE supports more advanced interactive visualization
* Added Search visualization API to work with the re-designed search


Tutorial
* Added new tutorial about debugging in FE to help people debug FE easier
* Updated the search tutorial to leverage new visualization API
* Added a new model robustness tutorial to showcase model robustness testing with Search API and Visualization API

Others:
* Significantly reduced the time needed to import FE through lazy import
* Added dependency warning check to warn inconsistent backend versions

This is a patch release that fixes outdated dependency: sklearn and pycocotool for FE 1.4.X

This release includes a hot-fix for a multiprocessing issue with mac book

History API
* Fixed a critical bug about database access

Backend
* Fixed a bug caused by assertion during relaxed shape static graph execution
* Hardware-agnostic pytorch model saving & loading

Dataset
* New dataset - skl digits

Network
* New Permute TensorOp
* New Normalize TensorOp

Architecture & Apphubs
* Momentum inconsistency between TensorFlow and PyTorch

Estimator
* New per-instance CSV metric reporting and its per-instance interface for custom Trace
* BleuScore Trace as NLP metric

Plotting
* Fixed several bugs in log plotting

XAI
* New Eigen Cam capability: allowing user to specify percentage of variation