Tensorflow-model-analysis

Major Features and Improvements

* Removes the metrics modules from experimental now that it is migrated to
[py-ml-metrics](https://pypi.org/project/py-ml-metrics/) package.
* Adds Constituent Flip Rate Metrics: SymmetricFlipRate, NegToNegFlipRate,
NegToPosFlipRate, PosToNegFlipRate, PosToPosFlipRate.
* Depend on tensorflow-estimator package explicitly.

Bug fixes and other Changes

* Fix the bug about batching unsized numpy arrays.

Breaking Changes

* Removes `attrs` requirement.
* Consolidate Matrix definition for semantic segmentation confusion matrix
metrics.
* Provide AggregateFn and interface and default call impl to adapt TFMA
metrics combiner for in-process call.
* Move Mean metrics from experimental to metrics.
* Fix the bug of size estimator failure.
* Depends on `tensorflow>=2.15.0,<2.16`.
* Fix the failure in testMeanAttributions.
* Fix the input type mismatch in metric_specs_tests between bool and None.
* Fix the failure in the slice test due to beam type hints check.
* Fix the failure in metric_specs test, all TFMA deps on keras are
keras 2.
* Depends on `apache-beam[gcp]>=2.53.0,<3` for Python 3.11 and on
`apache-beam[gcp]>=2.47.0,<3` for 3.9 and 3.10.
* Depends on `protobuf>=4.25.2,<5` for Python 3.11 and on `protobuf>3.20.3,<5`
for 3.9 and 3.10.
* Update the minimum Bazel version required to build TFMA to 6.1.0
* Refactors BooleanFlipRates computations to a combiner (flip_counts) and a
DerivedMetricComputation (flip_rates).

Deprecations

* Deprecated python 3.8 support.

Major Features and Improvements

* Add F1, False positive rate, and Accuracy into the confusion matrix plot.
* Add support for setting top_k and class_id at the same time for confusion
matrix metrics.
* Add the false positive for semantic segmentation metrics.
* Add Mean Metric (experimental) which calculates the mean of any feature. *.
Adds support of `output_keypath` to ModelSignatureDoFn to explicitly set a
chain of output keys in the multi-level dict (extracts). Adds output_keypath
to common prediction extractors.
* Add Mean Metric (experimental) which calculates the mean of any feature. *.
Adds support of `output_keypath` to ModelSignatureDoFn to explicitly set a
chain of output keys in the multi-level dict (extracts). Adds output_keypath
to common prediction extractors.
* Add ROUGE Metrics.
* Add BLEU Metric.
* Refactor Binary Confusion Matrices to use Binary Confusion Matrices
Computations.

Bug fixes and other Changes

* Fix the bug that SetMatchRecall is always 1 when top_k is set.
* Depends on `pyarrow>=10,<11`.
* Depends on `apache-beam>=2.47,<3`.
* Depends on `numpy>=1.23.0`.
* Depends on `tensorflow>=2.13.0,<3`.
* Add 'tfma_eval' model_type in model_specs as the identifier for
eval_saved_model, allowing signature='eval' to now be used with other model
types.
* Add "materialized_prediction" model type to allow users bypassing model
inference explicitly.

Breaking Changes

* Depend on PIL for image related metrics.
* Separate extract_key from signature names in `ModelSignaturesDoFn`.

Deprecations

* N/A

Major Features and Improvements

* Add BinaryCrossEntropy and CategoricalCrossEntropy.
* Add MeanAbsolutePercentageError and MeanSquaredLogarithmicError
* Add SetMatchPrecision and SetMatchRecall
* Add SemanticSegmentationConfusionMatrix

Bug fixes and other Changes

* Fix for jupiter notebook
* Fix element dimension inconsistency when some of the extracts have missing
key.
* Add public visibility to the servo beam extractor.
* Fix for bug where binary_confusion_matrices with different class_weights are
considered identical and deduplicated.
* Fixes bug where positive / negative axes labels are reversed in prediction
distribution plot.
* Depends on `numpy~=1.22.0`.
* Modify ExampleCount to not depend on labels and predictions.
* Add class_id info into sub_key in metric_key for object detection confusion
matrix metrics.
* Add class_id info into sub_key in plot_key for object detection confusion
matrix plot.
* Fix a bug that auto_pivot dropped nan when deciding which columns are
multivalent for pivoting.
* Depends on `tensorflow>=2.12.0,<2.13`.
* Depends on `protobuf>=3.20.3,<5`.
* Depends on `tfx-bsl>=1.13.0,<1.14.0`.
* Depends on `tensorflow-metadata>=1.13.1,<1.14.0`.
* Add name for each plots.

Breaking Changes

* N/A

Deprecations

Deprecated python3.7 support.

Major Features and Improvements

* N/A

Bug fixes and other Changes

* Depends on `tensorflow>=2.11,<3`
* Depends on `tfx-bsl>=1.2.0,<1.13.0`.
* Depends on `tensorflow-metadata>=1.12.0,<1.13.0`.

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support
will be removed in the next version. Please check the
[TF2 migration guide](https://www.tensorflow.org/guide/migrate) to migrate
to TF2.
* Add BooleanFlipRate metric for comparing thresholded predictions between
multiple models.
* Add CounterfactualPredictionsExtractor for computing predictions on modified
inputs.
* Add MeanAbsoluteError and MeanSquaredError

Bug fixes and other Changes

* Add support for parsing the Predict API prediction log output to the
experimental TFX-BSL PredictionsExtractor implementation.
* Add support for parsing the Classification API prediction log output to the
experimental TFX-BSL PredictionsExtractor implementation.
* Update remaining predictions_extractor_test.py tests to cover
PredictionsExtractorOSS. Fixes a pytype bug related to multi tensor output.
* Depends on `tensorflow>=1.15.5,<2` or `tensorflow>=2.10,<3`

* Apply changes in the latest Chrome browser

* Add InferneceInterface to experimental PredictionsExtractor implementation.

* Stop returning empty example_ids metric from binary_confusion_matrices
derived computations when example_id_key is not set but use_histrogam is
true.

* Add transformed features lookup for NDCG metrics query key and gain key.

* Deprecate BoundedValue and TDistribution in ConfusionMatrixAtThresholds.

* Fix a bug that dataframe auto_pivot fails if there is only Overall slice.

* Use SavedModel PB to determine default signature instead of loading the
model.

* Reduce clutter in the multi-index columns and index in the experimental
dataframe auto_pivot util.

* Minor predictions_extractor_test.py refactor with readability improvements
and improved test coverage.

Breaking Changes

* N/A

Deprecations

* N/A

Major Features and Improvements

* N/A

Bug fixes and other Changes

* Move the version to top of init.py since the original "from
tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

* N/A

Deprecations

* N/A