Artifacts
- Support for artifact tagging
**SDK**: Add `tag_artifacts` and `delete_artifacts_tags` that can be used to modify existing artifacts tags and have more than one version for an artifact.
**API**: Introduce new endpoints in `/projects/<project>/tags`.
Auth
- Support S3 profile and assume-role when using `fsspec`.
- Support GitHub fine grained tokens.
Functions
- Add `function.with_annotations({"framework":"tensorflow"})` to user created functions.
- Add `overwrite_build_params` to `project.build_function()` so the user can choose whether or not to keep the build params that were used in previous function builds.
Feature Store
- Support Redis as an online feature set, for storey engine only.
- Support GCP objects as a data source for the feature store.
- Fully support ingesting with pandas engine - now equivalent to ingestion with `storey` engine:
Support DataFrame with multi-index.
Support mlrun steps when using pandas engine: `OneHotEncoder` , `DateExtractor`, `MapValue`, `Imputer` and `FeatureValidation`.
- Add new step: `DropFeature` for pandas and storey engines.
- Add param query for `get_offline_feature` for filtering the output.
Frameworks
- Add `HuggingFaceModelServer` to `mlrun.frameworks` at `mlrun.frameworks.huggingface` to serve `HuggingFace` models.
Installation
- Add option to install `google-cloud` requirements using `mlrun[google-cloud]`: when installing MLRun for integration with GCP clients, only compatible packages are installed.
Documentation
- Restructured, and new content
Third party integrations
- Supports Confluent Kafka (Tech Preview)
Internal
- Refactor artifacts endpoints to follow the MLRun convention of `/projects/<project>/artifacts/...` .
- Add `/api/_internal/memory-reports/` endpoints for memory related metrics to better understand the memory consumption of the API.
- Improve the HTTP retry mechanism.
- Support a new lightweight mechanism for KFP pods to pull the run state they triggered. Default behavior is legacy, which pulls the logs of the run to figure out the run state. The new behavior can be enabled using a feature flag configured in the API.
Breaking changes
- Feature store: Ingestion using pandas now takes the dataframe and creates an index out of the entity column (and removes it from
being a column in this df). This could cause breakage for existing custom steps when using a pandas engine.
Bug fixes:
- Support logging artifacts larger than 5GB to V3IO. [2455](https://github.com/mlrun/mlrun/issues/2455)
- Limit KFP to `kfp~=1.8.0, <1.8.14` due to non-backwards changes done in `1.8.14` for `ParallelFor`, which isn’t compatible with the MLRun managed KFP server (1.8.1). [2516](https://github.com/mlrun/mlrun/issues/2516)
- Add `artifact_path` enrichment from project artifact_path . Previously, the parameter wasn't applied to project runs when defining `project.artifact_path`. [2507](https://github.com/mlrun/mlrun/issues/2507)
- Align timeouts for requests that are getting re-routed from worker to chief (for projects/background related endpoints). [2565](https://github.com/mlrun/mlrun/issues/2565)
- Fix legacy artifacts load when loading a project. Fixed corner cases when legacy artifacts were saved to yaml and loaded back into the system using `load_project()`. [2584](https://github.com/mlrun/mlrun/issues/2584)
- Fix artifact `latest` tag enrichment to happen also when user defined a specific tag. [2572](https://github.com/mlrun/mlrun/issues/2572)
- Fix `zip` source extraction during function build. [2588](https://github.com/mlrun/mlrun/issues/2588)
- Fix Docker compose deployment so Nuclio is configured properly with a platformConfig file that sets proper mounts and network configuration for Nuclio functions, meaning that they run in the same network as MLRun. [2601](https://github.com/mlrun/mlrun/issues/2601)
- Workaround for background tasks getting cancelled prematurely, due to the current FastAPI version that has a bug in the starlette package it uses. The bug caused the task to get cancelled if the client’s http connection was closed before the task was done. [2618](https://github.com/mlrun/mlrun/issues/2618)
- Fix run fails after deploying function without defined image. [2530](https://github.com/mlrun/mlrun/pull/2530).
- Scheduled jobs failed on GKE with resource quota error. [2520](https://github.com/mlrun/mlrun/pull/2520).
- Can now delete a model via tag. [2433](https://github.com/mlrun/mlrun/pull/2433).