Overview
Incremental training
Estimators are now re-trainable on new data, using the `train_from` method. This accepts a previously trained model (predictor), and new data to train on, and can greatly reduce training time if combined with early stopping. The feature is integrated with `gluonts.shell`-based SageMaker containers, and can be used by specifying the additional `model` channel to point to the output of a previous training job. More info in 2249.
New models
Two models are added in this release:
* `DeepVARHierarchicalEstimator`, a hierarchical extension to `DeepVAREstimator`; learn more about how to use this in [this tutorial](https://ts.gluon.ai/dev/tutorials/forecasting/hierarchical_model_tutorial.html).
* `DeepNPTSEstimator`, a global extension to NPTS, where sampling probabilities are learned from data; learn more on how to use this estimator [here](https://ts.gluon.ai/dev/api/gluonts/gluonts.torch.model.deep_npts.html).
Deprecated import paths and options
This release moves MXNet-based models from `gluonts.model` to `gluonts.mx.model`; the old import paths continue working in this release, but are deprecated and will be removed in the next release. For example, now the MXNet-based `DeepAREstimator` should be imported from `gluonts.mx` (or `gluonts.mx.model.deepar`).
We also removed deprecated options for learning rate reduction in the `gluonts.mx.Trainer` class: these can now be controlled via the `LearningRateReduction` callback.
Dataset splitting functionality (experimental)
We updated the functionality to split time series datasets (along the time axis) for training/validation/test purposes. Now this functionality can be easily accessed via the `split` function (`from gluonts.dataset.split import split`); learn more about this [here](https://ts.gluon.ai/dev/tutorials/data_manipulation/dataset_splitting_example.html).
This feature is experimental and subject to future changes.
Changelog
Breaking changes
* Breaking: Update data splitters to return `(input, output)` pairs in the test split (2031 by npnv)
* Breaking: Move MXNet-based models to `mx.model`. (2126 by Hongqing-work)
* Convert time-features into functions. (2149 by jaheba)
* Remove deprecated args from `mx.Trainer`. (2153 by jaheba)
* Reduce sdist size. (2199 by jaheba)
* Remove core.exception module. (2202 by jaheba)
* Remove core.ty. (2203 by jaheba)
* Update `gluonts.dataset.split` code, test, docs (2223 by lostella)
* Remove gluonts_forecasters entrypoint mechanic. (2278 by jaheba)
* Enable 'python -m gluonts'. (2292 by jaheba)
New features / major improvements
* Interrupting `mx.Trainer` stops training. (2131 by Hongqing-work)
* Expose evaluator aggregation_strategy functions (2198 by kashif)
* Add data preparation utility for hierarchical time series and a tutorial notebook (2206 by rshyamsundar)
* Add Deep NPTS model (1835 by rshyamsundar)
* Improve arrow reading performance. (2217 by mr-1993)
* Allow DeepVAR model to use (global) dynamic features (2226 by rshyamsundar)
* Hierarchical: Allow use of external dynamic features and add a section in the tutorial (2253 by rshyamsundar)
* Add `serde.dataclass`. (2166 by jaheba)
* R: Add Python wrapper for calling R's hierarchical methods (1685 by rshyamsundar)
* Add learning rate and weight decay arguments to PyTorch estimators (2289)
* Added LR scheduler to DeepAR Pytorch (2287 by shubhamkapoor)
* Add LR scheduling patience option to MQF2 (2291 by lostella)
* Add incremental training (2249 by lostella)
* Add input size and type information to DeepARModel, and example_input_array to DeepARLightningModule. (2307 by jgasthaus)
* Add `dataset.schema.translate`. (2304 by jaheba)
* Add `forecast_start` to entry-wise metrics in evaluator (2312 by lostella)
Bug fixes / minor improvements
* Fix DatasetCollection (2135 by rsnirwan)
* Fix `PandasDataset` for Python 3.9 (2141 by lostella)
* Make `PandasDataset` faster (2148 by lostella)
* Ignore divide warnings in evaluation. (2159 by jaheba)
* Fix Prophet wrapper to work with `Timestamp` instead of `Period` (2182 by lostella)
* Fix dtype for "item_id" column in metrics dataframe (2183 by lostella)
* Fix recursive case for `gluonts.mx.batchify.stack` (2184 by lostella)
* Fix `item_id` values in `ConstantValuePredictor` (2192 by codingWhale13)
* Fixup Patience class. (2197 by jaheba)
* Fix dataset arrow writer tool. (2196 by jaheba)
* Fix `SymbolBlock` serde issue (2187 by lostella)
* Add item id to Uber TLC dataset (2214 by mvanness354)
* Fix `r_forecast` wrapper to shift start date when truncating time series (2216 by abdulfatir)
* Fix dtype bug in piecewise_linear and add a test (2224 by rshyamsundar)
* Fix bug in `to_quantile_forecast` (2225 by eugeneteoh)
* Fix `gluonts.mx.trainer.Trainer` in case of empty data loader (2228 by lostella)
* Fix feed-forward models when features are provided (2238 by lostella)
* update SplicedBinnedPareto demos from nursery version to gluonts version (2250 by elenaehrlich)
* Improve len() for ParquetFile. (2261 by jaheba)
* Move max_idle_transform usage to GluonEstimator. (2262 by jaheba)
* Optimize `TimeSeriesSlice` performance (2259 by lostella)
* Fix ignore hidden files when generating datasets (2263 by kashif)
* Fix: set max idle transforms in PyTorch estimators (2266 by lostella)
* Fix `QuantileForecast.plot()` to use `DateTimeIndex` (2269 by abdulfatir)
* Fix serde dataclass eventual. (2277 by jaheba)
* Fix `gluonts.dataset.split` for multivariate case (2314 by lostella)
* Improve `TestData` class in `gluonts.dataset.split` (2315 by lostella)
* Simplify `make_evaluation_predictions` (2309 by lostella)
* Fix MQCNN for `kernel_size=1` (2321 by lostella)
* Simplify unbatching in forecast-generator. (2334 by jaheba)
* Fix numerical bug in `BinnedUniforms` (2344 by moudheus)
Documentation
* Docs: Make notebook templates. (2122 by jaheba)
* Docs: Rework `installation` section. (2130 by jaheba)
* Docs: Fix running tutorials for publishing docs. (2138 by jaheba)
* Docs: Update hyperparameter tuning with optuna notebook. (2137 by npnv)
* Fix issues with hyperparameter tuning tutorial (2143 by lostella)
* Apply black to notebooks. (2144 by jaheba)
* Docs: Simplify wide `DataFrame` example (2150 by lostella)
* Docs: fix links in models table (2156 by lostella)
* Add 'Background' section to docs. (2129 by jaheba)
* Docs: Add info about version guarantees. (2161 by jaheba)
* Docs: fix tutorial after breaking changes in trainer class (2179 by lostella)
* Add tutorial with data splitting examples (2157 by npnv)
* Fix: add missing link to splitting tutorial (2185 by lostella)
* Fix: ensure last cell of tutorials runs (2186 by lostella)
* Fixes to the dataset splitting tutorial (2189 by npnv)
* Update TSBench readme with paper reference (2191 by geoalgo)
* Update Available models table with the hierarchical model (2209 by rshyamsundar)
* Fix broken links in Available-models table (2211 by rshyamsundar)
* Add logo to README. (2248 by jaheba)
* New logo. (2243 by jaheba)
* Use brand colors in docs. (2257 by jaheba)
* Docs: Reformatting table, badge colors. (2258 by jaheba)
* Docs: update contribution guidelines and dev setup (2270 by lostella)
* Add Github footer icon to docs. (2285 by jaheba)
* Docs: Custom Pygments style for dark theme. (2290 by jaheba)
* Fix README quick examples (2297 by lostella)
* Fix text in Quick Start Tutorial (2300 by sighellan)
* Update README and tutorial (2311 by lostella)
* Turn on apidoc generation (2332 by jaheba)
* Add info on how to use 'just' (2339 by codingWhale13)
* Small documentation improvements (2343 by codingWhale13)
Test / setup changes
* add python 3.9 to test workflows (2136)
* Tests: Move mx model test. (2158 by jaheba)
* Test: Use spawn method for shell server tests. (2177 by jaheba)
* Remove `holidays` and `matplotlib` from core dependencies. (2055 by jaheba)
* Update minimal version for nbconvert. (2233 by jaheba)
* Hierarchical: Add a test for to_dataset method (2265 by rshyamsundar)
* Fix mypy and black commands in pre-commit githook (2271 by abdulfatir)
* Update project_urls. (2274 by jaheba)
* Move _version to meta. (2293 by jaheba)
* Remove setup-requires. (2295 by jaheba)
* Remove pytest.ini. (2298 by jaheba)
* Speed up smoke tests (2341 by lostella)