Tensorflow-transform

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Depends on `apache-beam[gcp]>=2.29,<3`.
* Depends on
`tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,!=2.4.*,<2.6`.
* Depends on `tensorflow-metadata>=1.0.0,<1.1.0`.
* Depends on `tfx-bsl>=1.0.0,<1.1.0`.

Breaking Changes

* `tft.ptransform_analyzer` has been moved under `tft.experimental`. The order
of args in the API has also been changed.
* `tft_beam.PTransformAnalyzer` has been moved under `tft_beam.experimental`.
* The default value of the `drop_unused_features` parameter to
`TFTransformOutput.transform_raw_features` is now True.

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug Fixes and Other Changes

* Removed the `dataset_schema` module, most methods in it have been deprecated
since version 0.14.
* Fix a bug where having an analyzer operate on the output of `tft.vocabulary`
would cause it to evaluate incorrectly when `force_tf_compat_v1=False` with
TF2 behaviors enabled.
* Depends on `tensorflow-metadata>=0.30.0,<0.31.0`.
* Depends on `tfx-bsl>=0.30.0,<0.31.0`.

Breaking Changes

* `DatasetMetadata` no longer accepts a dict as its input schema. `schema` is
expected to be a `Schema` proto now.
* TF 1.15 specific APIs `apply_saved_model` and
`apply_function_with_checkpoint` were removed from the `tft` namespace. They
are still available under the `pretrained_models` module.
* `tft.AnalyzeDataset`, `tft.AnalyzeDatasetWithCache`,
`tft.AnalyzeAndTransformDataset` and `tft.TransformDataset` will use the
native TF2 implementation of tf.transform unless TF2 behaviors are
explicitly disabled. The previous behaviour can still be obtained by setting
`tft.Context.force_tf_compat_v1=True`.

Deprecations

* N/A

Major Features and Improvements

* `tft.AnalyzeAndTransformDataset` and `tft.TransformDataset` can now output
`pyarrow.RecordBatch`es. This is controlled by a parameter
`output_record_batches` which is set to `False` by default.

Bug Fixes and Other Changes

* Added `tft.make_and_track_object` to load and track `tf.Trackable` objects
created inside the `preprocessing_fn` (for example, tf.hub models). This API
should only be used when `force_tf_compat_v1=False` and TF2 behavior is
enabled.
* The `decode` method of the available coders (`tft.coders.CsvCoder` and
`tft.coders.ExampleProtoCoder`) have been removed. These were deprecated in
the 0.25 release.
[Canned TFXIO implementations](https://www.tensorflow.org/tfx/tfx_bsl/api_docs/python/tfx_bsl/public/tfxio)
should be used to read and decode data instead.
* Previously deprecated APIs were removed: `tft.uniques` (replaced by
`tft.vocabulary`), `tft.string_to_int` (replaced by
`tft.compute_and_apply_vocabulary`), `tft.apply_vocab` (replaced by
`tft.apply_vocabulary`), and `tft.apply_function` (identity function).
* Removed the `always_return_num_quantiles` arg of `tft.quantiles` and
`tft.bucketize` which was deprecated in version 0.26.
* Added support for `count_params` method to the `TransformFeaturesLayer`.
This will allow to call Keras Model's `summary()` method if the model is
using the `TransformFeaturesLayer`.
* Depends on `absl-py>=0.9,<0.13`.
* Depends on `tensorflow-metadata>=0.29.0,<0.30.0`.
* Depends on `tfx-bsl>=0.29.0,<0.30.0`.

Breaking Changes

* Existing caches (for all analyzers) are automatically invalidated.

Deprecations

* N/A

Major Features and Improvements

* Large vocabularies are now computed faster due to partially parallelizing
`VocabularyOrderAndWrite`.

Bug Fixes and Other Changes

* Generic `tf.SparseTensor` input support has been added to
`tft.scale_to_0_1`, `tft.scale_to_z_score`, `tft.scale_by_min_max`,
`tft.min`, `tft.max`, `tft.mean`, `tft.var`, `tft.sum`, `tft.size` and
`tft.word_count`.
* Optimize SavedModel written out by `tf.Transform` when using native TF2 to
speed up loading it.
* Added `tft_beam.PTransformAnalyzer` as a base PTransform class for
`tft.ptransform_analyzer` users who wish to have access to a base temporary
directory.
* Fix an issue where >2D `SparseTensor`s may be incorrectly represented in
instance_dicts format.
* Added support for out-of-vocabulary keys for per_key mappers.
* Added `tft.get_num_buckets_for_transformed_feature` which provides the
number of buckets for a transformed feature if it is a direct output of
`tft.bucketize`, `tft.apply_buckets`, `tft.compute_and_apply_vocabulary` or
`tft.apply_vocabulary`.
* Depends on `apache-beam[gcp]>=2.28,<3`.
* Depends on `numpy>=1.16,<1.20`.
* Depends on `tensorflow-metadata>=0.28.0,<0.29.0`.
* Depends on `tfx-bsl>=0.28.1,<0.29.0`.

Breaking changes

* Autograph is disabled when the preprocessing fn is traced using tf.function
when `force_tf_compat_v1=False` and TF2 behavior is enabled.

Deprecations

Major Features and Improvements

* Added `QuantilesCombiner.compact` method that moves some amount of work done
by `tft.quantiles` from non-parallelizable to parallelizable stage of the
computation.

Bug Fixes and Other Changes

* Strip only newlines instead of all whitespace in the TFTransformOutput
vocabulary_by_name method.
* Switch analyzers that output asset files to return an eager tensor
containing the asset file path instead of a tf.saved_model.Asset object when
`force_tf_compat_v1=False`. If this file is then used to initialize a table,
this ensures the input to the `tf.lookup.TextFileInitializer` is the file
path as the initializer handles wrapping this in a `tf.saved_model.Asset`
object.
* Added `tft.annotate_asset` for annotating asset files with a string key that
can be used to retrieve them in `tft.TFTransformOutput`.
* Depends on `apache-beam[gcp]>=2.27,<3`.
* Depends on `pyarrow>=1,<3`.
* Depends on `tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.*,<2.5`.
* Depends on `tensorflow-metadata>=0.27.0,<0.28.0`.
* Depends on `tfx-bsl>=0.27.0,<0.28.0`.

Breaking changes

* N/A

Deprecations

* Parameter `use_tfxio` in the initializer of `Context` is removed (it was
deprecated in 0.24.0).

Major Features and Improvements

* Initial support added of >2D `SparseTensor`s as inputs and outputs of the
`preprocessing_fn`. Note that mappers and analyzers may not support those
yet, and output >2D `SparseTensor`s will have an unknown dense shape.

Bug Fixes and Other Changes

* Switched to calling tables and initializers within `tf.init_scope` when the
`preprocessing_fn` is traced using `tf.function` to avoid re-initializing
them on every invocation of the traced `tf.function`.
* Switched to a (notably) faster and more accurate implementation of
`tft.quantiles` analyzer.
* Fix an issue where graphs become non-hermetic if a TF2 transform_fn is
loaded in a TF1 Graph context, by making sure all assets are added to the
`ASSET_FILEPATHS` collection.
* Depends on `apache-beam[gcp]>=2.25,!=2.26.*,<3`.
* Depends on `pyarrow>=0.17,<0.18`.
* Depends on `tensorflow>=1.15.2,!=2.0.*,!=2.1.*,!=2.2.*,<2.4`.
* Depends on `tensorflow-metadata>=0.26.0,<0.27.0`.
* Depends on `tfx-bsl>=0.26.0,<0.27.0`.

Breaking changes

* Existing `tft.quantiles`, `tft.min` and `tft.max` caches are invalidated.

Deprecations

* Parameter `always_return_num_quantiles` of `tft.quantiles` and
`tft.bucketize` is now deprecated. Both now always generate the requested
number of buckets. Setting `always_return_num_quantiles` will have no effect
and it will be removed in the next version.