Tensorflow-data-validation

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `tensorflow>=2.15,<2.16`.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* When computing cross feature statistics, skip configured crosses that
include features of unsupported types (i.e., are not univalent numeric
features).
* Update the minimum Bazel version required to build TFDV to 6.1.0.
* Modifies get_statistics_html() utility function to return a value indicating
a dataset has no examples.
* Outputs both a standard and a quantiles histogram for level N value list
length statistics.
* Add a `macos_arm64` config setting to the TFDV build file. NOTE: At this
time, any M1 support for TFDV is experimental and untested.
* Bumps the pybind11 version to 2.11.1.
* Depends on `tensorflow~=2.15.0`.
* Depends on `apache-beam[gcp]>=2.53.0,<3` for Python 3.11 and on
`apache-beam[gcp]>=2.47.0,<3` for 3.9 and 3.10.
* Depends on `protobuf>=4.25.2,<5` for Python 3.11 and on `protobuf>3.20.3,<5`
for 3.9 and 3.10.

Known Issues

* N/A

Breaking Changes

* N/A

Deprecations

* Deprecated python 3.8 support.
* Deprecated Windows support.

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Bumped the Ubuntu version on which TFX-BSL is tested to 20.04 (previously
was 16.04).
* Use platforms instead of bazel_tools//platforms to specify constraints in
OSS build.
* Depends on `pyarrow>=10,<11`.
* Depends on `apache-beam>=2.47,<3`.
* Depends on `numpy>=1.22.0`.
* Depends on `tensorflow>=2.13.0,<3`.

Known Issues

* N/A

Breaking Changes

* Moves some non-public arrow_util functions to TFX-BSL.
* Changes SkewPair proto to store tf.Examples in serialized format.

Deprecations

* N/A

Major Features and Improvements

* Introduces a Schema option `HistogramSelection` to allow numeric drift/skew
calculations to use QUANTILES histograms, which are more robust to outliers.

Bug Fixes and Other Changes

* Rename `statistics_io_impl` and `default_record_sink` (not part of public API).
* Update the minimum Bazel version required to build TFDV to 5.3.0.
* Depends on `numpy~=1.22.0`.
* Depends on `pyfarmhash>=0.2.2,<0.4`.
* Depends on `tensorflow>=2.12.0,<2.13`.
* Depends on `protobuf>=3.20.3,<5`.
* Depends on `tfx-bsl>=1.13.0,<1.14.0`.
* Depends on `tensorflow-metadata>=1.13.1,<1.14.0`.

Known Issues

* N/A

Breaking Changes

* Jensen-Shannon divergence now treats NaN values as always contributing to
higher drift score.

Deprecations

* Deprecated python 3.7 support.

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* TFDV is now tested against macOS 12.5 (Monterey).

Known Issues

* N/A

Breaking Changes

* Depends on `tensorflow>=2.11,<3`
* Depends on `tfx-bsl>=1.12.0,<1.13.0`.
* Depends on `tensorflow-metadata>=1.12.0,<1.13.0`.

Deprecations

* N/A

Major Features and Improvements

* This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support
will be removed in the next version. Please check the
[TF2 migration guide](https://www.tensorflow.org/guide/migrate) to migrate
to TF2.

* Add a `custom_validate_statistics` function to the validation API, and
support passing custom validations to `validate_statistics`. Note that
custom validation is not supported on Windows.

Bug Fixes and Other Changes

* Fix bug in implementation of `semantic_domain_stats_sample_rate`.

* Add beam metrics on string length

* Determine whether to calculate string statistics based on the
`is_categorical` field in the schema string domain.

* Histograms counts should now be more accurate for distributions with few
distinct values, or frequent individual values.

* Nested list length histogram counts are no longer based on the number of
values one up in the nested list hierarchy.

* Support using jensen-shannon divergence to detect drift and skew for string
and categorical features.

* `get_drift_skew_dataframe` now includes a `threshold` column.

* Adds support for NormalizedAbsoluteDifference comparator.

* Depends on `tensorflow>=1.15.5,<2` or `tensorflow>=2.10,<3`
* Depends on `joblib>=1.2.0`.

Known Issues

* N/A

Breaking Changes

* Histogram semantics are slightly changed, so that buckets include their
upper bound instead of their lower bound. STANDARD histograms will no longer
generate buckets that contain infinite and finite endpoints together.
* Introduces StatsOptions.use_sketch_based_topk_uniques replacing
experimental_use_sketch_based_topk_uniques. The latter option can still be
written, but not read.

Deprecations

* N/A