Kartothek

===========================

New functionality
^^^^^^^^^^^^^^^^^
* Add :func:`~kartothek.io.dask.dataframe.store_dataset_from_ddf` to offer write
support of a dask dataframe without update support. This forbids or explicitly
allows overwrites and does not update existing datasets.
* The ``sort_partitions_by`` feature now supports multiple columns. While this
has only marginal effect for predicate pushdown, it may be used to improve the
parquet compression.
* ``build_cube_from_dataframe`` now supports the ``shuffle`` methods offered by
:func:`~kartothek.io.dask.dataframe.store_dataset_from_ddf` and
:func:`~kartothek.io.dask.dataframe.update_dataset_from_ddf` but writes the
output in the cube format

Improvements
^^^^^^^^^^^^
* Reduce memory consumption during index write.
* Allow `simplekv` stores and `storefact` URLs to be passed explicitly as input for the `store` arguments

===========================

New functionality
^^^^^^^^^^^^^^^^^
* Add ``hash_dataset`` functionality

Improvements
^^^^^^^^^^^^

* Expand ``pandas`` version pin to include 1.1.X
* Expand ``pyarrow`` version pin to include 1.x
* Large addition to documentation for multi dataset handling (Kartothek Cubes)

===========================

* Fix evaluation of "OR"-connected predicates (295)

===========================

Improvements
^^^^^^^^^^^^

* Update timestamp related code into Ktk Discover Cube functionality.
* Support backward compatibility to old cubes and fix for cli entry point.

===========================

New functionality
^^^^^^^^^^^^^^^^^

* Introduction of ``cube`` Functionality which is made with multiple Kartothek datasets.
* Basic Features - Extend, Query, Remove(Partitions),
Delete (can delete entire datasets/cube), API, CLI, Core and IO features.
* Advanced Features - Multi-Dataset with Single Table, Explicit physical Partitions, Seed based join system.

===========================

New functionality
^^^^^^^^^^^^^^^^^

* Add :meth:`~kartothek.io_components.metapartition.MetaPartition.get_parquet_metadata` and :func:`~kartothek.io.dask.dataframe.collect_dataset_metadata`, enabling users to collect information about the Parquet metadata of a dataset (306)

Bug fixes
^^^^^^^^^

* Performance of dataset update with ``delete_scope`` significantly improved for datasets with many partitions (308)