Prefect

Released on September 1, 2020.

Enhancements

- Begin storing the width of mapped pipelines on the parent Mapped state - [3233](https://github.com/PrefectHQ/prefect/issues/3233)
- Kubernetes agent now manages lifecycle of prefect jobs in its namespace - [3158](https://github.com/PrefectHQ/prefect/pull/3158)
- Move agent heartbeat to background thread - [3158](https://github.com/PrefectHQ/prefect/pull/3158)
- Handles `ModuleNotFound` errors in the storage healthcheck - [3225](https://github.com/PrefectHQ/prefect/pull/3225)
- Raises the `warnings.warn` stack level to 2 to reduce duplicate warning messages - [3225](https://github.com/PrefectHQ/prefect/pull/3225)
- Add some extra output to the `client.register` print output for visibility - [3225](https://github.com/PrefectHQ/prefect/pull/3225)
- CLI help text docstrings are now auto documented using the API documentation parser - [3225](https://github.com/PrefectHQ/prefect/pull/3225)
- `DaskExecutor` now logs dask worker add/removal events - [3227](https://github.com/PrefectHQ/prefect/pull/3227)

Fixes

- Fix issue with passing --env-vars flag to K8s Agent Install manifest - [3239](https://github.com/PrefectHQ/prefect/issues/3239)
- Fix edge case with `add_edge` method - [3230](https://github.com/PrefectHQ/prefect/pull/3230)

Deprecations

- Kubernetes resource manager is now deprecated and the functionality is moved into the Kubernetes agent - [3158](https://github.com/PrefectHQ/prefect/pull/3158)

Contributors

- [shaunc](https://github.com/shaunc)

Released on August 25, 2020.

Enhancements

- Allow for setting path to a custom job YAML spec on the Kubernetes Agent - [3046](https://github.com/PrefectHQ/prefect/pull/3046)
- Use better coupled versioning scheme for Core / Server / UI images - [3204](https://github.com/PrefectHQ/prefect/pull/3204)
- Added option to mount volumes with KubernetesAgent - [1234](https://github.com/PrefectHQ/prefect/pull/3210)
- Add more kwargs to State.children and State.parents for common access patterns - [3212](https://github.com/PrefectHQ/prefect/pull/3212)
- Reduce size of `prefecthq/prefect` Docker image - [3215](https://github.com/PrefectHQ/prefect/pull/3215)

Task Library

- Add `DatabricksSubmitRun` task for submitting Spark jobs on Databricks - [3166](https://github.com/PrefectHQ/prefect/pull/3166)

Fixes

- Fix Apollo service error output while waiting for GraphQL service with `prefect server start` - [3150](https://github.com/PrefectHQ/prefect/pull/3150)
- Fix `--api` CLI option not being respected by agent Client - [3186](https://github.com/PrefectHQ/prefect/pull/3186)
- Fix state message when using targets - [3216](https://github.com/PrefectHQ/prefect/pull/3216)

Contributors

- [Ashton Sidhu](https://github.com/Ashton-Sidhu)
- [Chris Martin](https://github.com/d80tb7)
- [Juan Calderon-Perez](https://github.com/gabrielcalderon)
- [Thomas Frederik Hoeck](https://github.com/thomasfrederikhoeck)
- [VincentTNR](https://github.com/VincentTNR)
- [emcake](https://github.com/emcake)

Released on August 18, 2020.

Enhancements

- Make use of `kubernetes` extra logger in the `DaskKubernetesEnvironment` optional - [2988](https://github.com/PrefectHQ/prefect/issues/2988)
- Make Client robust to simplejson - [3151](https://github.com/PrefectHQ/prefect/issues/3151)
- Raise Warning instead of Exception during storage healthcheck when Result type is not provided - [3146](https://github.com/PrefectHQ/prefect/pull/3146)
- Add `server create-tenant` for creating a tenant on the server - [3147](https://github.com/PrefectHQ/prefect/pull/3147)
- Cloud logger now responds to logging level - [3179](https://github.com/PrefectHQ/prefect/pull/3179)

Task Library

- Add support for [`host_config`](https://docker-py.readthedocs.io/en/stable/api.html#docker.api.container.ContainerApiMixin.create_host_config) and arbitrary keyword arguments in `Docker` tasks - [3173](https://github.com/PrefectHQ/prefect/pull/3173)

Fixes

- Fix empty string `imagePullSecrets` issue on AKS by removing if not set - [3142](https://github.com/PrefectHQ/prefect/issues/3142)
- Fix querying for cached states with no `cache_key` - [3168](https://github.com/PrefectHQ/prefect/pull/3168)
- Fix access to `core_version` in Agent's `get_flow_run_command()` - [3177](https://github.com/PrefectHQ/prefect/pull/3177)

Breaking Changes

- DaskKubernetesEnvironment no longer logs Kubernetes errors by default - [2988](https://github.com/PrefectHQ/prefect/issues/2988)
- Logging level in Cloud now defaults to INFO - [3179](https://github.com/PrefectHQ/prefect/pull/3179)

Contributors

- [James Lamb](https://github.com/jameslamb)
- [Nelson Cornet](https://github.com/sk4la)
- [Zach Angell](https://github.com/zangell44)

Released on August 11, 2020.

Features

- Pandas DataFrame Serializer - [3020](https://github.com/PrefectHQ/prefect/pull/3020), [#2963](https://github.com/PrefectHQ/prefect/pull/2963), [#2917](https://github.com/PrefectHQ/prefect/issues/2917)

Enhancements

- Agents set flow run execution command based on flow's core version - [3113](https://github.com/PrefectHQ/prefect/pull/3113)
- Clean up extra labels on jobs created by Kubernetes agent - [3129](https://github.com/PrefectHQ/prefect/pull/3129)

Task Library

- Return `LoadJob` object in `BigQueryLoad` tasks - [3086](https://github.com/PrefectHQ/prefect/issues/3086)

Fixes

- Fix bug with `LocalDaskExecutor('processes')` that allowed tasks to be run multiple times in certain cases - [3127](https://github.com/PrefectHQ/prefect/pull/3127)
- Add toggle to bypass bug in `slack_notifier` that attempted to connect to backend even if the backend didn't exist - [3136](https://github.com/PrefectHQ/prefect/pull/3136)

Contributors

- [Andrew Schechtman-Rook](https://github.com/AndrewRook)
- [Franklin Winokur](https://github.com/fwinokur)
- [Fraznist](https://github.com/Fraznist)
- [Jackson Maxfield Brown](https://github.com/JacksonMaxfield)

Released on August 6, 2020.

Fixes

- Fix issue with 0.13.0 agents not able to run Flows registered with older Core versions - [3111](https://github.com/PrefectHQ/prefect/pull/3111)

Released on August 6, 2020.

Features

- Support cancellation of active flow runs - [2942](https://github.com/PrefectHQ/prefect/pull/2942)
- Add Webhook storage - [3000](https://github.com/PrefectHQ/prefect/pull/3000)

Enhancements

- Only supply versions when setting `SUBMITTED` and `RUNNING` states - [2730](https://github.com/PrefectHQ/prefect/issues/2730)
- Gracefully recover from version lock errors - [2731](https://github.com/PrefectHQ/prefect/issues/2731)
- Add `--ui-version` server start CLI option to run a specific UI image - [3087](https://github.com/PrefectHQ/prefect/pull/3087)
- Agent querying of flow runs now passes active tenant ID - [3087](https://github.com/PrefectHQ/prefect/pull/3087)
- Ignore calls to flow.register when parsing a flow using file based storage - [3051](https://github.com/PrefectHQ/prefect/issues/3051)

Task Library

- Allow idempotency keys in `FlowRunTask` when using server backend - [3006](https://github.com/PrefectHQ/prefect/issues/3006)
- Require project name in `FlowRunTask` when using server backend - [3006](https://github.com/PrefectHQ/prefect/issues/3006)

Fixes

- Fix use of absolute path in Docker storage on Windows - [3044](https://github.com/PrefectHQ/prefect/pull/3044)
- Determine if checkpointing is enabled from config set in the flow-runner process - [3085](https://github.com/PrefectHQ/prefect/pull/3085)
- Fix `--no-ui` server start CLI option still attempting to pull UI image - [3087](https://github.com/PrefectHQ/prefect/pull/3087)

Deprecations

- Deprecate `execute cloud-flow` CLI command in favor of `execute flow-run` - [3087](https://github.com/PrefectHQ/prefect/pull/3087)
- Deprecate `run server/cloud` CLI commands in favor of `run flow` - [3087](https://github.com/PrefectHQ/prefect/pull/3087)

Breaking Changes

- Move server and UI code out into separate repositories - [3087](https://github.com/PrefectHQ/prefect/pull/3087)
- Project names are now required when managing flows with the core server - [3087](https://github.com/PrefectHQ/prefect/pull/3087)

Contributors

- [James Lamb](https://github.com/jameslamb)
- [Pravin Dahal](https://github.com/pravindahal)
- [Panagiotis Simakis](https://github.com/sp1thas)