Tensorflow-data-validation

Major Features and Improvements

* Add support for detecting drift and distribution skew in numeric features.
* `tfdv.validate_statistics` now also reports the raw measurements of
distribution skew/drift (if any is done), regardless whether skew/drift is
detected. The report is in the `drift_skew_info` of the `Anomalies` proto
(return value of `validate_statistics`).
* From this release TFDV will also be hosting nightly packages on
https://pypi-nightly.tensorflow.org. To install the nightly package use the
following command:


pip install --extra-index-url https://pypi-nightly.tensorflow.org/simple tensorflow-data-validation


Note: These nightly packages are unstable and breakages are likely to
happen. The fix could often take a week or more depending on the complexity
involved for the wheels to be available on the PyPI cloud service. You can
always use the stable version of TFDV available on PyPI by running the
command `pip install tensorflow-data-validation` .

Bug Fixes and Other Changes

* Added `tfdv.load_stats_binary` to load stats what were written using
`tfdv.WriteStatisticsToText` (now `tfdv.WriteStatisticsToBinaryFile`).
* Anomalies previously (un)classified as UKNOWN_TYPE now trigger more specific
anomaly types: DOMAIN_INVALID_FOR_TYPE, UNEXPECTED_DATA_TYPE,
FEATURE_MISSING_NAME, FEATURE_MISSING_TYPE, INVALID_SCHEMA_SPECIFICATION
* Fixed a bug that `import tensorflow_data_validation` would fail if IPython
is not installed. IPython is an optional dependency of TFDV.
* Depends on `apache-beam[gcp]>=2.25,<3`.
* Depends on `tensorflow-metadata>=0.25,<0.26`.
* Depends on `tensorflow-transform>=0.25,<0.26`.
* Depends on `tfx-bsl>=0.25,<0.26`.
* Depends on `scikit-learn>=1.0,<2` (mutual-information installation).

Known Issues

* N/A

Breaking Changes

* `tfdv.WriteStatisticsToText` is renamed as
`tfdv.WriteStatisticsToBinaryFile`. The former is still available but will
be removed in a future release.

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.24,<3`.
* Depends on `tensorflow-transform>=0.24.1,<0.25`.
* Depends on `tfx-bsl>=0.24.1,<0.25`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* You can now build the TFDV wheel with `python setup.py bdist_wheel`. Note:
* If you want to build a manylinux2010 wheel you'll still need
to use Docker.
* Bazel is still required.
* You can now build manylinux2010 TFDV wheel for Python 3.8.

Bug Fixes and Other Changes

* Support allowlist and denylist features in `tfdv.visualize_statistics`
method.
* Depends on `absl-py>=0.9,<0.11`.
* Depends on `pandas>=1.0,<2`.
* Depends on `protobuf>=3.9.2,<4`.
* Depends on `tensorflow-metadata>=0.24,<0.25`.
* Depends on `tensorflow-transform>=0.24,<0.25`.
* Depends on `tfx-bsl>=0.24,<0.25`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Deprecated Py3.5 support.
* Deprecated `sample_count` option in `tfdv.StatsOptions`. Use `sample_rate`
option instead.

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.24,<3`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Deprecating python 3.5 support.

Major Features and Improvements

* Data validation is now able to handle arbitrarily nested arrow
List/LargeList types. Schema entries for features with multiple nest levels
describe the value count at each level in the value_counts field.
* Add combiner stats generator to estimate top-K and uniques using Misra-Gries
and K-Minimum Values sketches.

Bug Fixes and Other Changes

* Validate that enough supported images are present (if
image_domain.minimum_supported_image_fraction is provided).
* Stopped requiring avro-python3.
* Depends on `apache-beam[gcp]>=2.23,<3`.
* Depends on `pyarrow>=0.17,<0.18`.
* Depends on `tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,<3`.
* Depends on `tensorflow-metadata>=0.23,<0.24`.
* Depends on `tensorflow-transform>=0.23,<0.24`.
* Depends on `tfx-bsl>=0.23,<0.24`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Note: We plan to remove Python 3.5 support after this release.

Major Features and Improvements

Bug Fixes and Other Changes
* Fixed a bug that affected tfx 0.22.0 to work with TFDV 0.22.1.
* Depends on 'avro-python3>=1.8.1,<1.9.2' on Python 3.5 + MacOS

Known Issues

Breaking Changes

Deprecations