Autogluon

We're happy to announce the AutoGluon 0.8 release.

NEW: [](https://discord.gg/wjUmjqAc2N) Join our official community discord server to ask questions and get involved!

Note: Loading models trained in different versions of AutoGluon is not supported.

This release contains 196 commits from 20 contributors!

See the full commit change-log here: https://github.com/autogluon/autogluon/compare/0.7.0...0.8.0

Special thanks to geoalgo for the joint work in generating the experimental tabular Zeroshot-HPO portfolio this release!

Full Contributor List (ordered by of commits):

shchur, Innixma, yinweisu, gradientsky, FANGAreNotGnu, zhiqiangdon, gidler, liangfu, tonyhoo, cheungdaven, cnpgs, giswqs, suzhoum, yongxinw, isunli, jjaeyeon, xiaochenbin9527, yzhliu, jsharpna, sxjscience

AutoGluon 0.8 supports Python versions 3.8, 3.9, and 3.10.

Changes

Highlights
* AutoGluon TimeSeries introduced several major improvements, including new models, upgraded presets that lead to better forecast accuracy, and optimizations that speed up training & inference.
* AutoGluon Tabular now supports **[calibrating the decision threshold in binary classification](https://auto.gluon.ai/stable/tutorials/tabular/tabular-indepth.html#decision-threshold-calibration)** ([API](https://auto.gluon.ai/stable/api/autogluon.tabular.TabularPredictor.calibrate_decision_threshold.html)), leading to massive improvements in metrics such as `f1` and `balanced_accuracy`. It is not uncommon to see `f1` scores improve from `0.70` to `0.73` as an example. We **strongly** encourage all users who are using these metrics to try out the new decision threshold calibration logic.
* AutoGluon MultiModal introduces two new features: 1) [**PDF document classification**](https://auto.gluon.ai/stable/tutorials/multimodal/document/pdf_classification.html), and 2) [**Open Vocabulary Object Detection**](https://auto.gluon.ai/stable/tutorials/multimodal/object_detection/quick_start/quick_start_ovd.html).
* AutoGluon MultiModal upgraded the presets for object detection, now offering `medium_quality`, `high_quality`, and `best_quality` options. The empirical results demonstrate significant ~20% relative improvements in the mAP (mean Average Precision) metric, using the same preset.
* AutoGluon Tabular has added an experimental **Zeroshot HPO config** which performs well on small datasets <10000 rows when at least an hour of training time is provided (~60% win-rate vs `best_quality`). To try it out, specify `presets="experimental_zeroshot_hpo_hybrid"` when calling `fit()`.
* AutoGluon EDA added support for [**Anomaly Detection**](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-anomaly-detection.html) and [**Partial Dependence Plots**](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-analyze-interaction.html#using-interaction-charts-to-learn-information-about-the-data).
* AutoGluon Tabular has added experimental support for **[TabPFN](https://github.com/automl/TabPFN)**, a pre-trained tabular transformer model. Try it out via `pip install autogluon.tabular[all,tabpfn]` (hyperparameter key is "TABPFN")! You can also try it out via specifying `presets="experimental_extreme_quality"`.

General
* General doc improvements tonyhoo Innixma yinweisu gidler cnpgs isunli giswqs (2940, 2953, 2963, 3007, 3027, 3059, 3068, 3083, 3128, 3129, 3130, 3147, 3174, 3187, 3256, 3258, 3280, 3306, 3307, 3311, 3313)
* General code fixes and improvements yinweisu Innixma (2921, 3078, 3113, 3140, 3206)
* CI improvements yinweisu gidler yzhliu liangfu gradientsky (2965, 3008, 3013, 3020, 3046, 3053, 3108, 3135, 3159, 3283, 3185)
* New AutoGluon Webpage gidler shchur (2924)
* Support sample_weight in RMSE jjaeyeon (3052)
* Move AG search space to common yinweisu (3192)
* Deprecation utils yinweisu (3206, 3209)
* Update namespace packages for PEP420 compatibility gradientsky (3228)

Multimodal

AutoGluon MultiModal (also known as AutoMM) introduces two new features: 1) PDF document classification, and 2) Open Vocabulary Object Detection. Additionally, we have upgraded the presets for object detection, now offering `medium_quality`, `high_quality`, and `best_quality` options. The empirical results demonstrate significant ~20% relative improvements in the mAP (mean Average Precision) metric, using the same preset.

New Features
* PDF Document Classification. See [tutorial](https://auto.gluon.ai/stable/tutorials/multimodal/document/pdf_classification.html) cheungdaven (#2864, 3043)
* Open Vocabulary Object Detection. See [tutorial](https://auto.gluon.ai/stable/tutorials/multimodal/object_detection/quick_start/quick_start_ovd.html) FANGAreNotGnu (#3164)

Performance Improvements
* Upgrade the detection engine from mmdet 2.x to mmdet 3.x, and upgrade our presets FANGAreNotGnu (3262)
* `medium_quality`: yolo-s -> yolox-l
* `high_quality`: yolox-l -> DINO-Res50
* `best_quality`: yolox-x -> DINO-Swin_l
* Speedup fusion model training with deepspeed strategy. liangfu (2932)
* Enable detection backbone freezing to boost finetuning speed and save GPU usage FANGAreNotGnu (3220)

Other Enhancements
* Support passing data path to the fit() API zhiqiangdon (3006)
* Upgrade TIMM to the latest v0.9.* zhiqiangdon (3282)
* Support xywh output for object detection FANGAreNotGnu (2948)
* Fusion model inference acceleration with TensorRT liangfu (2836, 2987)
* Support customizing advanced image data augmentation. Users can pass a list of [torchvision transform](https://pytorch.org/vision/stable/transforms.html#geometry) objects as image augmentation. zhiqiangdon (3022)
* Add yoloxm and yoloxtiny FangAreNotGnu (3038)
* Add MultiImageMix Dataset for Object Detection FangAreNotGnu (3094)
* Support loading specific checkpoints. Users can load the intermediate checkpoints other than model.ckpt and last.ckpt. zhiqiangdon (3244)
* Add some predictor properties for model statistics zhiqiangdon (3289)
* `trainable_parameters` returns the number of trainable parameters.
* `total_parameters` returns the number of total parameters.
* `model_size` returns the model size measured by megabytes.

Bug Fixes / Code and Doc Improvements
* General bug fixes and improvements zhiqiangdon liangfu cheungdaven xiaochenbin9527 Innixma FANGAreNotGnu gradientsky yinweisu yongxinw (2939, 2989, 2983, 2998, 3001, 3004, 3006, 3025, 3026, 3048, 3055, 3064, 3070, 3081, 3090, 3103, 3106, 3119, 3155, 3158, 3167, 3180, 3188, 3222, 3261, 3266, 3277, 3279, 3261, 3267)
* General doc improvements suzhoum (3295, 3300)
* Remove clip from fusion models liangfu (2946)
* Refactor inferring problem type and output shape zhiqiangdon (3227)
* Log GPU info including GPU total memory, free memory, GPU card name, and CUDA version during training zhiqaingdon (3291)


Tabular

New Features
* Added `calibrate_decision_threshold` ([tutorial](https://auto.gluon.ai/stable/tutorials/tabular/tabular-indepth.html#decision-threshold-calibration)), which allows to optimize a given metric's decision threshold for predictions to strongly enhance the metric score. Innixma (3298)
* We've added an experimental Zeroshot HPO config, which performs well on small datasets <10000 rows when at least an hour of training time is provided. To try it out, specify `presets="experimental_zeroshot_hpo_hybrid"` when calling `fit()` Innixma geoalgo (3312)
* The [TabPFN model](https://auto.gluon.ai/stable/api/autogluon.tabular.models.html#tabpfnmodel) is now supported as an experimental model. TabPFN is a viable model option when inference speed is not a concern, and the number of rows of training data is less than 10,000. Try it out via `pip install autogluon.tabular[all,tabpfn]`! Innixma (3270)
* Backend support for distributed training, which will be available with the next Cloud module release. yinweisu (3054, 3110, 3115, 3131, 3142, 3179, 3216)
Performance Improvements
* Accelerate boolean preprocessing Innixma (2944)
Other Enhancements
* Add quantile regression support for CatBoost shchur (3165)
* Implement quantile regression for LGBModel shchur (3168)
* Log to file support yinweisu (3232)
* Add support for `included_model_types` yinweisu (3239)
* Add enable_categorical=True support to XGBoost Innixma (3286)
Bug Fixes / Code and Doc Improvements
* Cross-OS loading of a fit TabularPredictor should now work properly yinweisu Innixma
* General bug fixes and improvements Innixma cnpgs shchur yinweisu gradientsky (2865, 2936, 2990, 3045, 3060, 3069, 3148, 3182, 3199, 3226, 3257, 3259, 3268, 3269, 3287, 3288, 3285, 3293, 3294, 3302)
* Move interpretable logic to InterpretableTabularPredictor Innixma (2981)
* Enhance drop_duplicates, enable by default Innixma (3010)
* Refactor params_aux & memory checks Innixma (3033)
* Raise regression `pred_proba` Innixma (3240)


TimeSeries
In v0.8 we introduce several major improvements to the Time Series module, including new models, upgraded presets that lead to better forecast accuracy, and optimizations that speed up training & inference.

Highlights
- New models: `PatchTST` and `DLinear` from GluonTS, and `RecursiveTabular` based on integration with the [`mlforecast`](https://github.com/Nixtla/mlforecast) library shchur (#3177, 3184, 3230)
- Improved accuracy and reduced overall training time thanks to updated presets shchur (3281, 3120)
- 3-6x faster training and inference for `AutoARIMA`, `AutoETS`, `Theta`, `DirectTabular`, `WeightedEnsemble` models shchur (3062, 3214, 3252)

New Features
- Dramatically faster repeated calls to `predict()`, `leaderboard()` and `evaluate()` thanks to prediction caching shchur (3237)
- Reduce overfitting by using multiple validation windows with the `num_val_windows` argument to `fit()` shchur (3080)
- Exclude certain models from presets with the `excluded_model_types` argument to `fit()` shchur (3231)
- New method `refit_full()` that refits models on combined train and validation data shchur (3157)
- Train multiple configurations of the same model by providing lists in the `hyperparameters` argument shchur (3183)
- Time limit set by `time_limit` is now respected by all models shchur (3214)

Enhancements
- Improvements to the `DirectTabular` model (previously called `AutoGluonTabular`): faster featurization, trained as a quantile regression model if `eval_metric` is set to `"mean_wQuantileLoss"` shchur (2973, 3211)
- Use correct seasonal period when computing the MASE metric shchur (2970)
- Check the AutoGluon version when loading `TimeSeriesPredictor` from disk shchur (3233)

Minor Improvements / Documentation / Bug Fixes
* Update documentation and tutorials shchur (2960, 2964, 3296, 3297)
* General bug fixes and improvements shchur (2977, 3058, 3066, 3160, 3193, 3202, 3236, 3255, 3275, 3290)

Exploratory Data Analysis (EDA) tools
In 0.8 we introduce a few new tools to help with data exploration and feature engineering:
* **Anomaly Detection** gradientsky (3124, 3137) - helps to identify unusual patterns or behaviors in data that deviate significantly from the norm. It's best used when finding outliers, rare events, or suspicious activities that could indicate fraud, defects, or system failures. Check the [Anomaly Detection Tutorial](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-anomaly-detection.html) to explore the functionality.
* **Partial Dependence Plots** gradientsky (3071, 3079) - visualize the relationship between a feature and the model's output for each individual instance in the dataset. Two-way variant can visualize potential interactions between any two features. Please see this tutorial for more detail: [Using Interaction Charts To Learn Information About the Data](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-analyze-interaction.html#using-interaction-charts-to-learn-information-about-the-data)
Bug Fixes / Code and Doc Improvements
* Switch regression analysis in `quick_fit` to use residuals plot gradientsky (3039)
* Added `explain_rows` method to `autogluon.eda.auto` - Kernel SHAP visualization gradientsky (3014)
* General improvements and fixes gradientsky (2991, 3056, 3102, 3107, 3138)

We're happy to announce the AutoGluon 0.7 release. This release contains a new experimental module `autogluon.eda` for exploratory
data analysis. AutoGluon 0.7 offers **conda-forge support**, enhancements to Tabular, MultiModal, and Time Series
modules, and many quality of life improvements and fixes.

As always, only load previously trained models using the same version of AutoGluon that they were originally trained on.
Loading models trained in different versions of AutoGluon is not supported.

This release contains [**170** commits from **19** contributors](https://github.com/autogluon/autogluon/graphs/contributors?from=2023-01-10&to=2023-02-16&type=c)!

See the full commit change-log here: https://github.com/autogluon/autogluon/compare/v0.6.2...v0.7.0

Special thanks to MountPOTATO who is a first time contributor to AutoGluon this release!

Full Contributor List (ordered by of commits):

Innixma, zhiqiangdon, yinweisu, gradientsky, shchur, sxjscience, FANGAreNotGnu, yongxinw, cheungdaven,
liangfu, tonyhoo, bryanyzhu, suzhoum, canerturkmen, giswqs, gidler, yzhliu, Linuxdex and MountPOTATO

AutoGluon 0.7 supports Python versions 3.8, 3.9, and **3.10**. Python 3.7 is no longer supported as of this release.

Changes

NEW: AutoGluon available on conda-forge

As of AutoGluon 0.7 release, AutoGluon is now available on [conda-forge](https://anaconda.org/conda-forge/autogluon) (#612)!

Kudos to the following individuals for making this happen:
* giswqs for leading the entire effort and being a 1-man army driving this forward.
* h-vetinari for providing excellent advice for working with conda-forge and some truly exceptional feedback.
* arturdaraujo, PertuyF, ngam and priyanga24 for their encouragement, suggestions, and feedback.
* The conda-forge team for their prompt and effective reviews of our (many) PRs.
* gradientsky for testing M1 support during the early stages.
* sxjscience, zhiqiangdon, canerturkmen, shchur, and Innixma for helping upgrade our downstream dependency versions to be compatible with conda.
* Everyone else who has supported this process either directly or indirectly.

NEW: `autogluon.eda` (Exploratory Data Analysis)

We are happy to announce AutoGluon Exploratory Data Analysis (EDA) toolkit. Starting with v0.7, AutoGluon now can analyze and visualize different aspects of data and models. We invite you to explore the following tutorials: [Quick Fit](https://auto.gluon.ai/dev/tutorials/stable/eda-auto-quick-fit.html), [Dataset Overview](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-dataset-overview.html), [Target Variable Analysis](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-target-analysis.html), [Covariate Shift Analysis](https://auto.gluon.ai/stable/tutorials/eda/eda-auto-covariate-shift.html). Other materials can be found in [EDA Section](https://auto.gluon.ai/stable/tutorials/eda/index.html) of the website.

General

- Added Python 3.10 support. Innixma (2721)
- Dropped Python 3.7 support. Innixma (2722)
- Removed `dask` and `distributed` dependencies. Innixma (2691)
- Removed `autogluon.text` and `autogluon.vision` modules. We recommend using `autogluon.multimodal` for text and vision tasks going forward.

AutoMM

AutoGluon MultiModal (a.k.a AutoMM) supports three new features: 1) document classification; 2) named entity recognition
for Chinese language; 3) few shot learning with SVM

Meanwhile, we removed `autogluon.text` and `autogluon.vision` as these features are supported in `autogluon.multimodal`

New features

- Document Classification
- Add scanned document classification (experimental).
- Customers can train models for scanned document classification in a few lines of codes
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/document/document_classification.html)
- Contributors and commits: cheungdaven (2765, 2826, 2833, 2928)
- NER for Chinese Language
- Support Chinese named entity recognition
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/document/document_classification.html)
- Contributors and commits: cheungdaven (2676, 2709)
- Few Shot Learning with SVM
- Improved few shot learning by adding SVM support
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/few_shot_learning.html)
- Contributors and commits: yongxinw (2850)

Other Enhancements

- Add new loss function `FocalLoss`. yongxinw (2860)
- Add matcher realtime inference support. zhiqiangdon (2613)
- Add matcher HPO. zhiqiangdon (2619)
- Add YOLOX models (small, large, and x-large) and update presets for object detection. FANGAreNotGnu (2644, 2867, 2927, 2933)
- Add AutoMM presets zhiqiangdon. (2620, 2749, 2839)
- Add model dump for models from HuggingFace, timm and mmdet. suzhoum FANGAreNotGnu liangfu (2682, 2700, 2737, 2840)
- Bug fix / refactor for NER. cheungdaven (2659, 2696, 2759, 2773)
- MultiModalPredictor import time reduction. sxjscience (2718)

Bug Fixes / Code and Doc Improvements

- NER example with visualization. sxjscience (2698)
- Bug fixes / Code and Doc Improvements. sxjscience tonyhoo giswqs (2708, 2714, 2739, 2782, 2787, 2857, 2818, 2858, 2859, 2891, 2918, 2940, 2906, 2907)
- Support of [Label-Studio](https://labelstud.io/) file export in AutoMM and added [examples](https://github.com/autogluon/autogluon/tree/master/examples/automm/label_studio_export_reader). MountPOTATO (#2615)
- Added example of few-shot memory bank model with feature extraction based on [Tip-adapter](https://arxiv.org/abs/2111.03930). Linuxdex (#2822)

Deprecations

* `autogluon.vision` namespace is deprecated. bryanyzhu (2790, 2819, 2832)
* `autogluon.text` namespace is deprecated. sxjscience Innixma (2695, 2847)

Tabular

1) TabularPredictor’s inference speed has been heavily optimized, with an average **250% speedup** for real-time inference. This means that TabularPredictor can satisfy <10 ms end-to-end latency on many datasets when using `infer_limit`, and the `high_quality` preset can satisfy <100 ms end-to-end latency on many datasets by default.
2) TabularPredictor’s `"multimodal"` hyperparameter preset now leverages the full capabilities of MultiModalPredictor, resulting in stronger performance on datasets containing a mix of tabular, image, and text features.

Performance Improvements

- Upgraded versions of all dependency packages to use the latest releases. Innixma (2823, 2829, 2834, 2887, 2915)
- Accelerated ensemble inference speed by 150% by removing TorchThreadManager context switching. liangfu (2472)
- Accelerated FastAI neural network inference speed by 100x+ and training speed by 10x on datasets with many features. Innixma (2909)
- (From 0.6.1) Avoid unnecessary DataFrame copies to accelerate feature preprocessing by 25%. liangfu (2532)
- (From 0.6.1) Refactor `NN_TORCH` model to be dataset iterable, leading to a 100% inference speedup. liangfu (2395)
- MultiModalPredictor is now used as a member of the ensemble when `TabularPredictor.fit` is passed `hyperparameters="multimodal"`. Innixma (2890)

API Enhancements

- Added `predict_multi` and `predict_proba_multi` methods to `TabularPredictor` to efficiently get predictions from multiple models. Innixma (2727)
- Allow label column to not be present in `leaderboard` calls when scoring is disabled. Innixma (2912)

Deprecations

- Added a deprecation warning when calling `predict_proba` with `problem_type="regression"`. This will raise an exception in a future release. Innixma (2684)

Bug Fixes / Doc Improvements

- Fixed incorrect time_limit estimation in `NN_TORCH` model. Innixma (2909)
- Fixed error when fitting with only text features. Innixma (2705)
- Fixed error when `calibrate=True, use_bag_holdout=True` in `TabularPredictor.fit`. Innixma (2715)
- Fixed error when tuning `n_estimators` with RandomForest / ExtraTrees models. Innixma (2735)
- Fixed missing onnxruntime dependency on Linux/MacOS when installing optional dependency `skl2onnx`. liangfu (2923)
- Fixed edge-case RandomForest error on Windows. yinweisu (2851)
- Added improved logging for `refit_full`. Innixma (2913)
- Added `compile_models` to the deployment tutorial. liangfu (2717)
- Various internal code refactoring. Innixma (2744, 2887)
- Various doc and logging improvements. Innixma (2668)

autogluon.timeseries

New features

- `TimeSeriesPredictor` now supports **past covariates** (a.k.a.dynamic features or related time series which is not known for time steps to be predicted). shchur (2665, 2680)
- New models from [StatsForecast](https://github.com/Nixtla/statsforecast) got introduced in `TimeSeriesPredictor` for various presets (`medium_quality`, `high_quality` and `best_quality`). shchur (#2758)
- Support missing value imputation for TimeSeriesDataFrame which allows users to customize filling logics for missing values and fill gaps in an irregular sampled times series. shchur (2781)
- Improve quantile forecasting performance of the AutoGluon-Tabular forecaster using the empirical noise distribution. shchur (2740)

Bug Fixes / Doc Improvements

- Bug fixes and code improvements. shchur canerturkmen (2703, 2712, 2713, 2769, 2771, 2816, 2817, 2875, 2877, 2919)
- Doc improvements. shchur gidler (2772, 2783, 2800)

v0.6.2 is a security and bug fix release.

As always, only load previously trained models using the same version of AutoGluon that they were originally trained on.
Loading models trained in different versions of AutoGluon is not supported.

See the full commit change-log here: https://github.com/autogluon/autogluon/compare/v0.6.1...v0.6.2

Special thanks to daikikatsuragawa and yzhliu who were first time contributors to AutoGluon this release!

This version supports Python versions 3.7 to 3.9. 0.6.x are the last releases that will support Python 3.7.

Changes

Documentation improvements

- Ray usage FAQ (2559) - yinweisu
- Fix missing Predictor API doc (2573) - gidler
- 2023 Roadmap Update (2590) - Innixma
- Image classifiction tutorial update for bytearray (2598) - suzhoum
- Fix broken tutorial index links (2617) - shchur
- Improve timeseries quickstart tutorial (2653) - shchur


Bug Fixes / Security

- [multimodal] Refactoring and bug fixes(2554, 2541, 2477, 2569, 2578, 2613, 2620, 2630, 2633, 2635, 2647, 2645, 2652, 2659) - zhiqiangdon, yongxinw, FANGAreNotGnu, sxjscience, Innixma
- [multimodal] Support of named entity recognition (2556) - cheungdaven
- [multimodal] bytearray support for image modality (2495) - suzhoum
- [multimodal] Support HPO for matcher (2619) - zhiqiangdon
- [multimodal] Support Onnx export for timm image model (2564) - liangfu
- [tabular] Refactoring and bug fixes (2387, 2595，2599, 2589, 2628, 2376, 2642, 2646, 2650, 2657) - Innixma, liangfu， yzhliu, daikikatsuragawa, yinweisu
- [tabular] Fix ensemble folding (2582) - yinweisu
- [tabular] Convert ColumnTransformer in tabular NN from sklearn to onnx (2503) - liangfu
- [tabular] Throw error on non-finite values in label column ($2509) - gidler
- [timeseries] Refactoring and bug fixes (2584, 2594, 2605, 2606) - shchur
- [timeseries] Speed up data preparation for local models (2587) - shchur
- [timeseries] Spped up prediction for GluonTS models (2593) - shchur
- [timeseries] Speed up the train/val splitter (2586) - shchur
[timeseries] Speed up TimeSeriesEnsembleSelection.fit (2602) - shchur
- [security] Update torch (2588) - gradientsky

v0.6.1 is a security fix / bug fix release.

As always, only load previously trained models using the same version of AutoGluon that they were originally trained on.
Loading models trained in different versions of AutoGluon is not supported.

See the full commit change-log here: https://github.com/autogluon/autogluon/compare/v0.6.0...v0.6.1

Special thanks to lvwerra who is first time contributors to AutoGluon this release!

This version supports Python versions 3.7 to 3.9. 0.6.x are the last releases that will support Python 3.7.

Changes

Documentation improvements

- Fix object detection tutorial layout (2450) - bryanyzhu
- Add multimodal cheatsheet (2467) - sxjscience
- Refactoring detection inference quickstart and bug fix on fit->predict - yongxinw, zhiqiangdon, Innixma, BingzhaoZhu, tonyhoo
- Use Pothole Dataset in Tutorial for AutoMM Detection (2468) - FANGAreNotGnu
- add time series cheat sheet, add time series to doc titles (2478) - canerturkmen
- Update all repo references to autogluon/autogluon (2463) - gidler
- fix typo in object detection tutorial CI (2516) - tonyhoo

Bug Fixes / Security

- bump evaluate to 0.3.0 (2433) - lvwerra
- Add finetune/eval tests for AutoMM detection (2441) - FANGAreNotGnu
- Adding Joint IA3_LoRA as efficient finetuning strategy (2451) - Raldir
- Fix AutoMM warnings about object detection (2458) - zhiqiangdon
- [Tabular] Speed up feature transform in tabular NN model (2442) - liangfu
- fix matcher cpu inference bug (2461) - sxjscience
- [timeseries] Silence GluonTS JSON warning (2454) - shchur
- [timeseries] Fix pandas groupby bug + GluonTS index bug (2420) - shchur
- Simplified infer speed throughput calculation (2465) - Innixma
- [Tabular] make tabular nn dataset iterable (2395) - liangfu
- Remove old images and dataset download scripts (2471) - Innixma
- Support image bytearray in AutoMM (2490) - suzhoum
- [NER] add an NER visualizer (2500) - cheungdaven
- [Cloud] Lazy load TextPredcitor and ImagePredictor which will be deprecated (2517) - tonyhoo
- Use detectron2 visualizer and update quickstart (2502) - yongxinw, zhiqiangdon, Innixma, BingzhaoZhu, tonyhoo
- fix df preprocessor properties (2512) - zhiqiangdon
- [timeseries] Fix info and fit_summary for TimeSeriesPredictor (2510) - shchur
- [timeseries] Pass known_covariates to component models of the WeightedEnsemble - shchur
- [timeseries] Gracefully handle inconsistencies in static_features provided by user - shchur
- [security] update Pillow to >=9.3.0 (2519) - gradientsky
- [CI] upgrade codeql v1 to v2 as v1 will be deprecated (2528) - tonyhoo
- Upgrade scikit-learn-intelex version (2466) - Innixma
- Save AutoGluonTabular model to the correct folder (2530) - shchur
- support predicting with model fitted on v0.5.1 (2531) - liangfu
- [timeseries] Implement input validation for TimeSeriesPredictor and improve debug messages - shchur
- [timeseries] Ensure that timestamps are sorted when creating a TimeSeriesDataFrame - shchur
- Add tests for preprocessing mutation (2540) - Innixma
- Fix timezone datetime edgecase (2538) - Innixma, gradientsky
- Mmdet Fix Image Identifier (2492) - FANGAreNotGnu
- [timeseries] Warn if provided data has a frequency that is not supported - shchur
- Train and inference with different image data types (2535) - suzhoum
- Remove pycocotools (2548) - bryanyzhu
- avoid copying identical dataframes (2532) - liangfu
- Fix AutoMM Tokenizer (2550) - FANGAreNotGnu
- [Tabular] Resource Allocation Fix (2536) - yinweisu
- imodels version cap (2557) - yinweisu
- Fix int32/int64 difference between windows and other platforms; fix mutation issue (2558) - gradientsky

We're happy to announce the AutoGluon 0.6 release. 0.6 contains major enhancements to Tabular, Multimodal, and Time Series
modules, along with many quality of life improvements and fixes.

As always, only load previously trained models using the same version of AutoGluon that they were originally trained on.
Loading models trained in different versions of AutoGluon is not supported.

This release contains [**263** commits from **25** contributors](https://github.com/awslabs/autogluon/graphs/contributors?from=2022-07-18&to=2022-11-15&type=c)!

See the full commit change-log here: https://github.com/awslabs/autogluon/compare/v0.5.2...v0.6.0

Special thanks to cheungdaven, suzhoum, BingzhaoZhu, liangfu, Harry-zzh, gidler, yongxinw, martinschaef,
giswqs, Jalagarto, geoalgo, lujiaying and leloykun who were first time contributors to AutoGluon this release!

Full Contributor List (ordered by of commits):

shchur, yinweisu, zhiqiangdon, Innixma, FANGAreNotGnu, canerturkmen, sxjscience, gradientsky, cheungdaven,
bryanyzhu, suzhoum, BingzhaoZhu, yongxinw, tonyhoo, liangfu, Harry-zzh, Raldir, gidler, martinschaef,
giswqs, Jalagarto, geoalgo, lujiaying, leloykun, yiqings

This version supports Python versions 3.7 to 3.9. This is the last release that will support Python 3.7.

Changes

AutoMM

AutoGluon Multimodal (a.k.a AutoMM) supports three new features: 1) object detection, 2) named entity recognition, and 3) multimodal matching. In addition, the HPO backend of AutoGluon Multimodal has been upgraded to ray 2.0. It also supports fine-tuning billion-scale FLAN-T5-XL model on a single AWS g4.2x-large instance with improved parameter-efficient finetuning. Starting from 0.6, we recommend using autogluon.multimodal rather than autogluon.text or autogluon.vision and added deprecation warnings.

New features

- Object Detection
- Add new problem_type `"object_detection"`.
- Customers can run inference with pretrained object detection models and train their own model with three lines of code.
- Integrate with [open-mmlab/mmdetection](https://github.com/open-mmlab/mmdetection), which supports classic detection architectures like Faster RCNN, and more efficient and performant architectures like YOLOV3 and VFNet.
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/object_detection/index.html) and [examples](https://github.com/awslabs/autogluon/tree/master/examples/automm/object_detection) for more detail.
- Contributors and commits: FANGAreNotGnu, bryanyzhu, zhiqiangdon, yongxinw, sxjscience, Harry-zzh (2025, 2061, 2131, 2181, 2196, 2215, 2244, 2265, 2290, 2311, 2312, 2337, 2349, 2353, 2360, 2362, 2365, 2380, 2381, 2391, 2393, 2400, 2419, 2421, 2063, 2104, 2411)

- Named Entity Recognition
- Add new problem_type `"ner"`.
- Customers can train models to extract named entities with three lines of code.
- The implementation supports any backbones in huggingface/transformer, including the recently [FLAN-T5 series](https://arxiv.org/abs/2210.11416) released by Google.
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/text_prediction/ner.html) for more detail.
- Contributors and commits: cheungdaven (2183, 2232, 2220, 2282, 2295, 2301, 2337, 2346, 2361, 2372, 2394, 2412)

- Multimodal Matching
- Add new problem_type `"text_similarity"`, `"image_similarity"`, `"image_text_similarity"`.
- Users can now extract semantic embeddings with pretrained models for text-text, image-image, and text-image matching problems.
- Moreover, users can further finetune these models with relevance data.
- The semantic text embedding model can also be combined with BM25 to form a hybrid indexing solution.
- Internally, AutoGluon Multimodal implements a twin-tower architecture that is flexible in the choice of backbones for each tower. It supports image backbones in TIMM, text backbones in huggingface/transformers, and also the CLIP backbone.
- See [tutorials](https://auto.gluon.ai/stable/tutorials/multimodal/matching/index.html) for more detail.
- Contributors and commits: zhiqiangdon FANGAreNotGnu cheungdaven suzhoum sxjscience bryanyzhu (1975, 1994, 2142, 2179, 2186, 2217, 2235, 2284, 2297, 2313, 2326, 2337, 2347, 2357, 2358, 2362, 2363, 2375, 2378, 2404, 2416, 2407, 2417)

- Miscellaneous minor fixes. cheungdaven FANGAreNotGnu geoalgo zhiqiangdon (2402, 2409, 2026, 2401, 2418)

Other Enhancements

- Fix the FT-Transformer implementation and support Fastformer. BingzhaoZhu yiqings (1958, 2194, 2251, 2344, 2379, 2386)
- Support finetuning billion-scale FLAN-T5-XL in a single AWS g4.2x-large instance via improved parameter-efficient finetuning. See [tutorial](https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/efficient_finetuning_basic.html). Raldir sxjscience (#2032, 2108, 2285, 2336, 2352)
- Upgrade multimodal HPO to use ray 2.0 and also add [new tutorial](https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/hyperparameter_optimization.html). yinweisu suzhoum bryanyzhu (#2206, 2341)
- Further improvement on model distillation. Add [example](https://github.com/awslabs/autogluon/tree/master/examples/automm/distillation) and [tutorial](https://auto.gluon.ai/stable/tutorials/multimodal/advanced_topics/model_distillation.html). FANGAreNotGnu sxjscience (#1983, 2064, 2397)
- Revise the default presets of AutoMM for image classification problems. bryanyzhu (2351)
- Support backend=“automm” in autogluon.vision. bryanyzhu (2316)
- Add deprecated warning to autogluon.vision and autogluon.text and point the usage to autogluon.multimodal. bryanyzhu sxjscience (2268, 2315)
- Examples about [Kaggle: Feedback Prize prediction competition](https://www.kaggle.com/competitions/feedback-prize-effectiveness). We created [a solution](https://www.kaggle.com/code/mountpotatoq/autogluon-finetune-solutions) with AutoGluon Multimodal that obtained 152/1557 in the public leaderboard and 170/1557 in the private leaderboard, which is among the top 12% participants. The solution is public days before the DDL of the competition and obtained more than 3000 views. suzhoum MountPOTATO (#2129, 2168, 2333)
* Improve native inference speed. zhiqiangdon (2051, 2157, 2161, 2171)
* Other improvements, security/bug fixes. zhiqiangdon sxjscience FANGAreNotGnu, yinweisu Innixma tonyhoo martinschaef giswqs tonyhoo (1980, 1987, 1989, 2003, 2080, 2018, 2039, 2058, 2101, 2102, 2125, 2135, 2136, 2140, 2141, 2152, 2164, 2166, 2192, 2219, 2250, 2257, 2280, 2308, 2315, 2317, 2321, 2356, 2388, 2392, 2413, 2414, 2417, 2426, 2028, 2382, 2415, 2193, 2213, 2230)
* CI improvements. yinweisu (1965, 1966, 1972, 1991, 2002, 2029, 2137, 2151, 2156, 2163, 2191, 2214, 2369, 2113, 2118)


Experimental Features

- Support 11B-scale model finetuning with DeepSpeed. Raldir (2032)
- Enable few-shot learning with 11B-scale model. Raldir (2197)
- ONNX export example of hf_text model. FANGAreNotGnu (2149)

Tabular

New features

- New experimental model `FT_TRANSFORMER`. bingzhaozhu, innixma (2085, 2379, 2389, 2410)
- You can access it via specifying the `FT_TRANSFORMER` key
in the `hyperparameters` dictionary or via `presets="experimental_best_quality"`.
- It is recommended to use GPU to train this model, but CPU training is also supported.
- If given enough training time, this model generally improves the ensemble quality.

- New experimental model compilation support via `predictor.compile_models()`. liangfu, innixma (2225, 2260, 2300)
- Currently only Random Forest and Extra Trees have compilation support.
- You will need to install extra dependencies for this to work: `pip install autogluon.tabular[all,skl2onnx]`.
- Compiling models dramatically speeds up inference time (~10x) when processing small batches of samples (<10000).
- Note that a known bug exists in the current implementation: Refitting models after compilation will fail
and cause a crash. To avoid this, ensure that `.compile_models` is called only at the very end.
- Added `predictor.clone(...)` method to allow perfectly cloning a predictor object to a new directory.
This is useful to preserve the state of a predictor prior to altering it
(such as prior to calling `.save_space`, `.distill`, `.compile_models`, or `.refit_full`. innixma (2071)
- Added simplified `num_gpus` and `num_cpus` arguments to `predictor.fit` to control total resources.
yinweisu, innixma (2263)
- Improved stability and effectiveness of HPO functionality via various refactors regarding our usage of ray.
yinweisu, innixma (1974, 1990, 2094, 2121, 2133, 2195, 2253, 2263, 2330)
- Upgraded dependency versions: XGBoost 1.7, CatBoost 1.1, Scikit-learn 1.1, Pandas 1.5, Scipy 1.9, Numpy 1.23.
innixma (2373)
- Added python version compatibility check when loading a fitted TabularPredictor.
Will now error if python versions are incompatible. innixma (2054)
- Added `fit_weighted_ensemble` argument to `predictor.fit`. This allows the user to disable the weighted ensemble.
innixma (2145)
- Added cascade ensemble foundation logic. innixma (1929)

Other Enhancements
- Improved logging clarity when using `infer_limit`. innixma (2014)
- Significantly improved HPO search space of XGBoost. innixma (2123)
- Fixed HPO crashing when tuning Random Forest, Extra Trees, or KNN. innixma (2070)
- Optimized roc_auc metric scoring speed by 7x. innixma (2318, 2331)
- Fixed bug with AutoMM Tabular model crashing if not trained last. innixma (2309)
- Refactored `Scorer` classes to be easier to use, plus added comprehensive unit tests for all metrics. innixma (2242)
- Sped up TextSpecial feature generation during preprocessing by 20% gidler (2095)
- imodels integration improvements Jalagarto (2062)
- Fix crash when calling feature importance in quantile_regression. leloykun (1977)
- Add FAQ section for missing value imputation. innixma (2076)
- Various minor fixes and cleanup innixma, yinweisu, gradientsky, gidler (1997, 2031, 2124, 2144, 2178, 2340, 2342, 2345, 2374, 2339,
2348, 2403, 1981, 1982, 2234, 2233, 2243, 2269, 2288, 2307, 2367, 2019)

Time Series

New features

- `TimeSeriesPredictor` now supports **static features** (a.k.a. time series metadata, static covariates) and **
time-varying covariates** (a.k.a. dynamic features or related time series). shchur canerturkmen (1986, 2238,
2276, 2287)
- AutoGluon-TimeSeries now uses **PyTorch** by default (for `DeepAR` and `SimpleFeedForward`), removing the dependency
on MXNet. canerturkmen (2074, 2205, 2279)
- New models! `AutoGluonTabular` relies on XGBoost, LightGBM and CatBoost under the hood via the `autogluon.tabular`
module. `Naive` and `SeasonalNaive` forecasters are simple methods that provide strong baselines with no increase in
training time. `TemporalFusionTransformerMXNet` brings the TFT transformer architecture to AutoGluon. shchur (2106,
2188, 2258, 2266)
- Up to 20x faster parallel and memory-efficient training for statistical (local) forecasting models like `ETS`, `ARIMA`
and `Theta`, as well as `WeightedEnsemble`. shchur canerturkmen (2001, 2033, 2040, 2067, 2072, 2073, 2180,
2293, 2305)
- Up to 3x faster training for GluonTS models with data caching. GPU training enabled by default on PyTorch models.
shchur (2323)
- More accurate validation for time series models with multi-window backtesting. shchur (2013, 2038)
- `TimeSeriesPredictor` now handles irregularly sampled time series with `ignore_index`. canerturkmen, shchur (1993,
2322)
- Improved and extended presets for more accurate forecasting. shchur (2304)
- 15x faster and more robust forecast evaluation with updates to `TimeSeriesEvaluator` shchur (2147, 2150)
- Enabled Ray Tune backend for hyperparameter optimization of time series models. shchur (2167, 2203)

More tutorials and examples

Improved documentation and new tutorials:

- Updated [Quickstart tutorial](https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-quickstart.html)
- New! [In-depth tutorial](https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-indepth.html)
- New! [Overview of available models and hyperparameters](https://auto.gluon.ai/stable/tutorials/timeseries/forecasting-model-zoo.html)
- Updated [API documentation](https://auto.gluon.ai/stable/api/autogluon.predictor.html#module-5)

shchur (2120, 2127, 2146, 2174, 2187, 2354)

Miscellaneous

shchur
- Deprecate passing quantile_levels to TimeSeriesPredictor.predict (2277)
- Use static features in GluonTS forecasting models (2238)
- Make sure that time series splitter doesn't trim training series shorter than prediction_length + 1 (2099)
- Fix hyperparameter overloading in HPO for time series models (2189)
- Clean up the TimeSeriesDataFrame public API (2105)
- Fix item order in GluonTS models predictions (2092)
- Implement hash_ts_dataframe_items (2060)
- Speed up TimeSeriesDataFrame.slice_by_timestep (2020)
- Speed up RandomForestQuantileRegressor and ExtraTreesQuantileRegressor (2204)
- Various backend enhancements / refactoring / cleanup (2314, 2294, 2292, 2278, 1985, 2398)

canerturkmen
- Increase the number of samples used by DeepAR at prediction time (2291)
- revise timeseries presets to minimum context length of 10 (2065)
- Fix timeseries daily frequency inferred period (2100)
- Various backend enhancements / refactoring / cleanup (2286, 2302, 2240, 2093, 2098, 2044, 2385, 2355, 2405)

v0.5.3 is a security hotfix release.

This release is **non-breaking** when upgrading from v0.5.0. As always, only load previously trained models using the same version of AutoGluon that they were originally trained on. Loading models trained in different versions of AutoGluon is not supported.

See the full commit change-log here: https://github.com/awslabs/autogluon/compare/v0.5.2...v0.5.3

This version supports Python versions 3.7 to 3.9.