Sapsan

Changes

Command Line Interface (CLI)
- New & Changed Commands
- `sapsan create --name {name}` or `sapsan create -n {name}` - creates a custom project template tree
- `sapsan test` - runs pytest to make sure Sapsan is working correctly
- `sapsan get_examples` - copies Sapsan's examples into the current working directory for easy access
- `sapsan --version` - to check the installed version
- Updated CLI options and --help

Testing
- moved tests to Sapsan's root folder, so they are always accessible when installed via pip
- notebook tests don't create a separate folder, but test on the existing example notebooks
- `sapsan test` allows for the user to run pytest tests to check if everything was installed correctly

Custom Estimators
- To get started, run: `sapsan create -n {name}`
- where `{name}` should be replaced with your custom project name
- Significantly simplified the template structure, making it much easier to navigate and get started
- cleaned up by removing all unnecessary templates, leaving a few scripts that allow to customize the estimator (i.e. ML network), Jupyter Notebook interface, and Dockerfile to easily share your project
- pre-filled all templates with the custom project name

Examples
- `sapsan get_examples`: copies examples into `./sapsan_examples`
- makes the example jupyter notebooks with various ML algorithms easily accessible

Other
- `sapsan` now has `__version__` attribute
- Updated the following sections in Documentation to reflect the CLI and template changes
- [Installation](https://github.com/pikarpov-LANL/Sapsan/wiki/installation)
- [Getting Started](https://github.com/pikarpov-LANL/Sapsan/wiki/getting-started)
- [Custom Estimators](https://github.com/pikarpov-LANL/Sapsan/wiki/Custom-Estimator)
- [Custom Docker](https://github.com/pikarpov-LANL/Sapsan/wiki/Custom-Docker)
- Minor comment updates to example notebooks

Changes

Testing
- notebook testing errors trigger pytest
- estimator loading testing is the same for both PyTorch and Scikit-learn

MLflow
- for evaluation, some model training pars and metrics are also recorded to make it easier to see with what model the prediction was done

[PICAE Estimator](https://github.com/pikarpov-LANL/Sapsan/wiki/Estimators#physics-informed-convolutional-autoencoder)
- removed redundant torch.device pass for PICAE estimator

Estimators
- simplified ModelConfig() in both included estimators and custom template
- ModelConfig now only has \_\_init\_\_
- moved load() to backend, removed to_dict()
- added load_config() under core Estimator class in models.py to enforce it to be included in custom estimators (most cases won't be affected, as it is part of the backend pytorch_estimator.py)
- changes reflected in the estimator template + wiki
- Save config now saves all input parameters instead of only the ones tracked by MLflow
- Loading is now consistent between PyTorch and scikit-learn based models

Other
- minor formatting fixes in examples
- minor bug fixes

Changes

There are no new features in this update, but numerous quality of life improvements and bug fixes.

Data Loader
- fixed multi-checkpoint loading for hdf5 module
- they are added as new batches to the loaded data np.array
- fixed splitting by batches
- the rounding error when computing the fraction to split into batches has been corrected
- default batch_num=1, hence batch_size = input_size if it is not specified (no need to specify the batching in the loader)
- fixed input data size processing, when sampling is not specified

Backend
- [FakeBackend](https://github.com/pikarpov-LANL/Sapsan/wiki/API-Reference#backend-tracking) is now the default if backend is not specified

Sampling
- no longer need to pass original input shape
- it is determined from the data itself
- added a warning on dealing with irregular shapes
- the function will try to match the requested shape, but if it can't - a warning is issued

Tensor Calculation
- fixed Tensor calculation indices
- added "only_x_components" flag
- at times full tensor is too taxing on memory

Requirements
- corrected installation requirements under python 3.8.10+

Changes

Parallel GPU
- Parallel GPU training is done via Distributed Data Parallel (DDP), through Catalyst
- Besides locally, capable of running on HPC
- Fixes in train and evaluation methods to account for DDP
- Added a [Parallel GPU tutorial on Wiki](https://github.com/pikarpov-LANL/Sapsan/wiki/parallel-GPU-Training)

Physics
- Gradient Model - full 3D tensor is now calculated, fixes to the algorithm
- Dynamic Smagorinsky Model - fully operational, outputs full 3D tensor
- Added `tensor()` function to calculate a stress tensor (see [API entry](https://github.com/pikarpov-LANL/Sapsan/wiki/API-Reference#physics))
- assertion checks in various functions to make the errors more intelligible

Docker
- Minor quality of life simplifications
- Added a [Docker tutorial on Wiki](https://github.com/pikarpov-LANL/Sapsan/wiki/Custom-Docker)

MLflow
- Updated compatibility with MLflow 18.0
- No need to restart kernel when re-running the model in jupyter notebooks
- Fixed compatibility with Catalist 21.7 logging
- Added [MLflow tutorial on Wiki](https://github.com/pikarpov-LANL/Sapsan/wiki/MLflow-Tracking)

GUI
- Fixed compatibility with Catalist 21.7 - correct logging and plotting

Other
- Expanded [Custom Estimator tutorial on Wiki](https://github.com/pikarpov-LANL/Sapsan/wiki/Custom-Estimator)
- Added [Community Guidelines](https://github.com/pikarpov-LANL/Sapsan/wiki/Community-Guidelines) to add new ML models and contribute to Sapsan
- Set up a jupyter notebook example on [Google Colab](https://github.com/pikarpov-LANL/Sapsan/wiki/web) to play around with Sapsan
- Minor bug fixes

Changes

- requirement updates
- simplified hierarchy
- compatibility with Catalyst 21.5+
- removed automatic PyTorch install: user will need to do it manually

Changes

- fixed PICAE in pypi