Great-expectations

* [FEATURE] `DataAssistantResult` plot multiple metrics per expectation (5556)
* [FEATURE] Enable passing "exact_estimation" boolean at `DataAssistant.run()` level (default value is True) (5744)
* [FEATURE] Example notebook for Onboarding DataAssistant - `postgres` (5776)
* [BUGFIX] dir update for data_assistant_result (5751)
* [BUGFIX] Fix docs_integration pipeline (5734)
* [BUGFIX] Patch flaky E2E Cloud test with randomized suite names (5752)
* [BUGFIX] Fix RegexPatternStringParameterBuilder to use legal character repetition. Remove median, mean, and standard deviation features from OnboardingDataAssistant "datetime_columns_rule" definition. (5757)
* [BUGFIX] Move `SuiteValidationResult.meta` validation id propogation before `ValidationOperator._run_action` (5760)
* [BUGFIX] Update "column.partition" Metric to handle DateTime Arithmetic Properly (5764)
* [BUGFIX] JSON-serialize RowCondition and enable IDDict to support comparison operations (5765)
* [BUGFIX] Insure all estimators properly handle datetime-float conversion (5774)
* [BUGFIX] Return appropriate subquery type to Query Metrics for SA version (5783)
* [DOCS] added guide how to use gx with emr serverless (5623) (thanks bvolodarskiy)
* [DOCS] DOC-362: how to choose between working with a single or multiple batches of data (5745)
* [MAINTENANCE] Temporarily xfail E2E Cloud tests due to Azure env var issues (5787)
* [MAINTENANCE] Add ids to `DataConnectorConfig` (5740)
* [MAINTENANCE] Rename GX Cloud "contract" resource to "checkpoint" (5748)
* [MAINTENANCE] Rename GX Cloud "suite_validation_result" resource to "validation_result" (5749)
* [MAINTENANCE] Store Refactor - cloud store return types & http-errors (5730)
* [MAINTENANCE] profile_numeric_columns_diff_expectation (5741) (thanks stevensecreti)
* [MAINTENANCE] Clean up type hints around class constructors (5738)
* [MAINTENANCE] invoke docker (5703)
* [MAINTENANCE] Add plist to build docker test image daily. (5754)
* [MAINTENANCE] opt-out type-checking (5713)
* [MAINTENANCE] Enable Algolia UI (5753)
* [MAINTENANCE] Linting & initial typing for data context (5756)
* [MAINTENANCE] Update `oneshot` estimator to `quantiles` estimator (5737)
* [MAINTENANCE] Update Auto-Initializing Expectations to use `exact` estimator by default (5759)
* [MAINTENANCE] Send a Gx-Version header set to __version__ in requests to cloud (5758)
* [MAINTENANCE] invoke docker --detach and more typing (5770)
* [MAINTENANCE] In ParameterBuilder implementations, enhance handling of numpy.ndarray metric values, whose elements are or can be converted into datetime.datetime type. (5771)
* [MAINTENANCE] Config/Schema round_tripping (5697)
* [MAINTENANCE] Add experimental label to MetricStore Doc (5782)
* [MAINTENANCE] Remove `GeCloudIdentifier` creation in `Checkpoint.run()` (5784)

* [FEATURE] Example notebooks for multi-batch Spark (5683)
* [FEATURE] Introduce top-level `default_validation_id` in `CheckpointConfig` (5693)
* [FEATURE] Pass down validation ids to `ExpectationSuiteValidationResult.meta` within `Checkpoint.run()` (5725)
* [FEATURE] Refactor data assistant runner to compute formal parameters for data assistant run method signatures (5727)
* [BUGFIX] Restored sqlite database for tests (5742)
* [BUGFIX] Fixing a typo in variable name for default profiler for auto-initializing expectation "expect_column_mean_to_be_between" (5687)
* [BUGFIX] Remove `resource_type` from call to `StoreBackend.build_key` (5690)
* [BUGFIX] Update how_to_use_great_expectations_in_aws_glue.md (5685) (thanks bvolodarskiy)
* [BUGFIX] Updated how_to_use_great_expectations_in_aws_glue.md again (5696) (thanks bvolodarskiy)
* [BUGFIX] Update how_to_use_great_expectations_in_aws_glue.md (5722) (thanks bvolodarskiy)
* [BUGFIX] Update aws_glue_deployment_patterns.py (5721) (thanks bvolodarskiy)
* [DOCS] added guide how to use great expectations with aws glue (5536) (thanks bvolodarskiy)
* [DOCS] Document the ZenML integration for Great Expectations (5672) (thanks stefannica)
* [DOCS] Converts broken ZenML md refs to Technical Tags (5714)
* [DOCS] How to create a Custom Query Expectation (5460)
* [MAINTENANCE] Pin makefun package to version range for support assurance (5746)
* [MAINTENANCE] s3 link for logo (5731)
* [MAINTENANCE] Assign `resource_type` in `InlineStoreBackend` constructor (5671)
* [MAINTENANCE] Add mysql client to Dockerfile.tests (5681)
* [MAINTENANCE] `RuleBasedProfiler` corner case configuration changes (5631)
* [MAINTENANCE] Update teams.yml (5684)
* [MAINTENANCE] Utilize `e2e` mark on E2E Cloud tests (5691)
* [MAINTENANCE] pyproject.tooml build-system typo (5692)
* [MAINTENANCE] expand flake8 coverage (5676)
* [MAINTENANCE] Ensure Cloud E2E tests are isolated to `gx-cloud-e2e` stage of CI (5695)
* [MAINTENANCE] Add usage stats and initial database docker tests to CI (5682)
* [MAINTENANCE] Add `e2e` mark to `pyproject.toml` (5699)
* [MAINTENANCE] Update docker readme to mount your repo over the builtin one. (5701)
* [MAINTENANCE] Combine packages `rule_based_profiler` and `rule_based_profiler.types` (5680)
* [MAINTENANCE] ExpectColumnValuesToBeInSetSparkOptimized (5702)
* [MAINTENANCE] expect_column_pair_values_to_have_difference_of_custom_perc… (5661) (thanks exteli)
* [MAINTENANCE] Remove non-docker version of CI tests that are now running in docker. (5700)
* [MAINTENANCE] Add back `integration` mark to tests in `test_datasource_crud.py` (5708)
* [MAINTENANCE] DEVREL-2289/Stale/Triage (5694)
* [MAINTENANCE] revert expansive flake8 pre-commit checking - flake8 5.0.4 (5706)
* [MAINTENANCE] Bugfix for `cloud-db-integration-pipeline` (5704)
* [MAINTENANCE] Remove pytest-azurepipelines (5716)
* [MAINTENANCE] Remove deprecation warning from `DataConnector`-level `batch_identifiers` for `RuntimeDataConnector` (5717)
* [MAINTENANCE] Refactor `AbstractConfig` to make `name` and `id_` consistent attrs (5698)
* [MAINTENANCE] Move CLI tests to docker (5719)
* [MAINTENANCE] Leverage `DataContextVariables` in `DataContext` hierarchy to automatically determine how to persist changes (5715)
* [MAINTENANCE] Refactor `InMemoryStoreBackend` out of `store_backend.py` (5679)
* [MAINTENANCE] Move compatibility matrix tests to docker (5728)
* [MAINTENANCE] Adds additional file extensions for Parquet assets (5729)
* [MAINTENANCE] MultiBatch SqlExample notebook Update. (5718)
* [MAINTENANCE] Introduce NumericRangeEstimator class hierarchy and encapsulate existing estimator implementations (5735)

* [FEATURE] Improve estimation histogram computation in NumericMetricRangeMultiBatchParameterBuilder to include both counts and bin edges (5628)
* [FEATURE] Enable retrieve by name for datasource with cloud store backend (5640)
* [FEATURE] Update `DataContext.add_checkpoint()` to ensure validations within `CheckpointConfig` contain ids (5638)
* [FEATURE] Add `expect_column_values_to_be_valid_crc32` (5580) (thanks sp1thas)
* [FEATURE] Enable showing expectation suite by domain and by expectation_type -- from DataAssistantResult (5673)
* [BUGFIX] Patch flaky E2E GX Cloud tests (5629)
* [BUGFIX] Pass `--cloud` flag to `dgtest-cloud-overrides` section of Azure YAML (5632)
* [BUGFIX] Remove datasource from config on delete (5636)
* [BUGFIX] Patch issue with usage stats sync not respecting usage stats opt-out (5644)
* [BUGFIX] SlackRenderer / EmailRenderer links to deprecated doc (5648)
* [BUGFIX] Fix table.head metric issue when using BQ without temp tables (5630)
* [BUGFIX] Quick bugfix on all profile numeric column diff bounds expectations (5651) (thanks stevensecreti)
* [BUGFIX] Patch bug with `id` vs `id_` in Cloud integration tests (5677)
* [DOCS] Fix a typo in batch_request_parameters variable (5612) (thanks StasDeep)
* [MAINTENANCE] CloudDataContext add_datasource test (5626)
* [MAINTENANCE] Update stale.yml (5602)
* [MAINTENANCE] Add `id` to `CheckpointValidationConfig` (5603)
* [MAINTENANCE] Better error message for RuntimeDataConnector for BatchIdentifiers (5635)
* [MAINTENANCE] type-checking round 2 (5576)
* [MAINTENANCE] minor cleanup of old comments (5641)
* [MAINTENANCE] add `--clear-cache` flag for `invoke type-check` (5639)
* [MAINTENANCE] Install `dgtest` test runner utilizing Git URL in CI (5645)
* [MAINTENANCE] Make comparisons of aggregate values date aware (5642)
* [MAINTENANCE] Add E2E Cloud test for `DataContext.add_checkpoint()` (5653)
* [MAINTENANCE] Use docker to run tests in the Azure CI pipeline. (5646)
* [MAINTENANCE] add new invoke tasks to `tasks.py` and create new file `usage_stats_utils.py` (5593)
* [MAINTENANCE] Don't include 'test-pipeline' in extras_require dict (5659)
* [MAINTENANCE] move tool config to pyproject.toml (5649)
* [MAINTENANCE] Refactor docker test CI steps into jobs. (5665)
* [MAINTENANCE] Only run Cloud E2E tests in primary pipeline (5670)
* [MAINTENANCE] Improve DateTime Conversion Candling in Comparison Metrics & Expectations and Provide a Clean Object Model for Metrics Computation Bundling (5656)
* [MAINTENANCE] Ensure that `id_` fields in Marshmallow schema serialize as `id` (5660)
* [MAINTENANCE] data_context initial type checking (5662)

* [FEATURE] Multi-Batch Example Notebook - SqlDataConnector examples (5575)
* [FEATURE] Implement "is_close()" for making equality comparisons "reasonably close" for each ExecutionEngine subclass (5597)
* [FEATURE] expect_profile_numeric_columns_percent_diff_(inclusive bounds) (5586) (thanks stevensecreti)
* [FEATURE] DataConnector Query enabled for `SimpleSqlDatasource` (5610)
* [FEATURE] Implement the exact metric range estimate for NumericMetricRangeMultiBatchParameterBuilder (5620)
* [FEATURE] Ensure that id propogates from RuleBasedProfilerConfig to RuleBasedProfiler (5617)
* [BUGFIX] Pass cloud base url to datasource store (5595)
* [BUGFIX] Temporarily disable Trino `0.315.0` from requirements (5606)
* [BUGFIX] Update _create_trino_engine to check for schema before creating it (5607)
* [BUGFIX] Support `ExpectationSuite` CRUD at `BaseDataContext` level (5604)
* [BUGFIX] Update test due to change in postgres stdev calculation method (5624)
* [BUGFIX] Patch issue with `get_validator` on Cloud-backed `DataContext` (5619)
* [MAINTENANCE] Add name and id to DatasourceConfig (5560)
* [MAINTENANCE] Clear datasources in `test_data_context_datasources` to improve test performance and narrow test scope (5588)
* [MAINTENANCE] Fix tests that rely on guessing pytest generated random file paths. (5589)
* [MAINTENANCE] Do not set google cloud credentials for lifetime of pytest process. (5592)
* [MAINTENANCE] Misc updates to `Datasource` CRUD on `DataContext` to ensure consistent behavior (5584)
* [MAINTENANCE] Add id to `RuleBasedProfiler` config (5590)
* [MAINTENANCE] refactor to enable customization of quantile bias correction threshold for bootstrap estimation method (5587)
* [MAINTENANCE] Ensure that `resource_type` used in `GeCloudStoreBackend` is converted to `GeCloudRESTResource` enum as needed (5601)
* [MAINTENANCE] Create datasource with id (5591)
* [MAINTENANCE] Enable Azure blob storage integration tests (5594)
* [MAINTENANCE] Increase expectation kwarg line stroke width (5608)
* [MAINTENANCE] Added Algolia Scripts (5544) (thanks devanshdixit)
* [MAINTENANCE] Handle `numpy` deprecation warnings (5615)
* [MAINTENANCE] remove approximate comparisons -- they will be replaced by estimator alternatives (5618)
* [MAINTENANCE] Making the dependency on dev-lite clearer (5514)
* [MAINTENANCE] Fix tests in tests/integration/profiling/rule_based_profiler/ and tests/render/renderer/ (5611)
* [MAINTENANCE] DataContext in cloud mode test add_datasource (5625)

* [FEATURE] Integrate `DataContextVariables` with `DataContext` (5466)
* [FEATURE] Add mostly to MulticolumnMapExpectation (5481)
* [FEATURE] [MAINTENANCE] Revamped expect_profile_numeric_columns_diff_between_exclusive_threshold_range (5493) (thanks stevensecreti)
* [FEATURE] [CONTRIB] expect_profile_numeric_columns_diff_(less/greater)_than_or_equal_to_threshold (5522) (thanks stevensecreti)
* [FEATURE] Provide methods for returning ExpectationConfiguration list grouped by expectation_type and by domain_type (5532)
* [FEATURE] add support for Azure authentication methods (5229) (thanks sdebruyn)
* [FEATURE] Show grouped sorted expectations by Domain and by expectation_type (5539)
* [FEATURE] Categorical Rule in VolumeDataAssistant Should Use Same Cardinality As Categorical Rule in OnboardingDataAssistant (5551)
* [BUGFIX] Handle "division by zero" in "ColumnPartition" metric when all column values are NULL (5507)
* [BUGFIX] Use string dialect name if not found in enum (5546)
* [BUGFIX] Add `try/except` around `DataContext._save_project_config` to mitigate issues with permissions (5550)
* [BUGFIX] Explicitly pass in mostly as 1 if not set in configuration. (5548)
* [BUGFIX] Increase precision for categorical rule for fractional comparisons (5552)
* [DOCS] DOC-340 partition local installation guide (5425)
* [DOCS] Add DataHub Ingestion docs (5330) (thanks maggiehays)
* [DOCS] toc update for DataHub integration doc (5518)
* [DOCS] Updating discourse to GitHub Discussions in Docs (4953)
* [MAINTENANCE] Clean up payload for `/data-context-variables` endpoint to adhere to desired chema (5509)
* [MAINTENANCE] DataContext Refactor: DataAssistants (5472)
* [MAINTENANCE] Ensure that validation operators are omitted from Cloud variables payload (5510)
* [MAINTENANCE] Add end-to-end tests for multicolumn map expectations (5517)
* [MAINTENANCE] Ensure that *_store_name attrs are omitted from Cloud variables payload (5519)
* [MAINTENANCE] Refactor `key` arg out of `Store.serialize/deserialize` (5511)
* [MAINTENANCE] Fix links to documentation (5177) (thanks andyjessen)
* [MAINTENANCE] Readme Update (4952)
* [MAINTENANCE] E2E test for `FileDataContextVariables` (5516)
* [MAINTENANCE] Cleanup/refactor prerequisite for group/filter/sort Expectations by domain (5523)
* [MAINTENANCE] Refactor `GeCloudStoreBackend` to use PUT and DELETE HTTP verbs instead of PATCH (5527)
* [MAINTENANCE] `/profiler` Cloud endpoint support (5499)
* [MAINTENANCE] Add type hints to `Store` (5529)
* [MAINTENANCE] Move MetricDomainTypes to core (it is used more widely now than previously). (5530)
* [MAINTENANCE] Remove dependency pins on pyarrow and snowflake-connector-python (5533)
* [MAINTENANCE] use invoke for common contrib/dev tasks (5506)
* [MAINTENANCE] Add snowflake-connector-python dependency lower bound. (5538)
* [MAINTENANCE] enforce pre-commit in ci (5526)
* [MAINTENANCE] Providing more robust error handling for determining `domain_type` of an `ExpectationConfiguration` object (5542)
* [MAINTENANCE] Remove extra indentation from store backend test (5545)
* [MAINTENANCE] Plot-level dropdown for `DataAssistantResult` display charts (5528)
* [MAINTENANCE] Make DataAssistantResult.batch_id_to_batch_identifier_display_name_map private (in order to optimize auto-complete for ease of use) (5549)
* [MAINTENANCE] Initial Dockerfile for running tests and associated README. (5541)
* [MAINTENANCE] Other dialect test (5547)

* [FEATURE] QueryExpectations (5223)
* [FEATURE] Control volume of metadata output when running DataAssistant classes. (5483)
* [BUGFIX] Snowflake Docs Integration Test Fix (5463)
* [BUGFIX] DataProfiler Linting Fix (5468)
* [BUGFIX] Update renderer snapshots with `None` values removed (5474)
* [BUGFIX] Rendering Test failures (5475)
* [BUGFIX] Update `dependency-graph` pipeline YAML to ensure `--spark` gets passed to `dgtest` (5477)
* [BUGFIX] Make sure the profileReport obj does not have defaultdicts (breaks gallery JSON) (5491)
* [BUGFIX] Use Pandas.isnull() instead of NumPy.isnan() to check for empty values in TableExpectation._validate_metric_value_between(), due to wider types applicability. (5502)
* [BUGFIX] Spark Schema has unexpected field for `spark.sql.warehouse.dir` (5490)
* [BUGFIX] Conditionally pop values from Spark config in tests (5508)
* [DOCS] DOC-349 re-write and partition interactive mode expectations guide (5448)
* [DOCS] DOC-344 partition data docs on s3 guide (5437)
* [DOCS] DOC-342 partition how to configure a validation result store in amazon s3 guide (5428)
* [DOCS] link fix in onboarding data assistant guide (5469)
* [DOCS] Integrate great-expectation with ydata-synthetic (4568) (thanks arunnthevapalan)
* [DOCS] Add 'test' extra to setup.py with docs (5415)
* [DOCS] DOC-343 partition how to configure expectation store for aws s3 guide (5429)
* [DOCS] DOC-357 partition the how to create a new checkpoint guide (5458)
* [DOCS] Remove outdated release process docs. (5484)
* [MAINTENANCE] Update `teams.yml` (5457)
* [MAINTENANCE] Clean up GitHub Actions (5461)
* [MAINTENANCE] Adds documentation and examples changes for snowflake connection string (5447)
* [MAINTENANCE] DOC-345 partition the connect to s3 cloud storage with Pandas guide (5439)
* [MAINTENANCE] Add unit and integration tests for Splitting on Mod Integer (5452)
* [MAINTENANCE] Remove `InlineRenderer` invocation feature flag from `ExpectationValidationResult` (5441)
* [MAINTENANCE] `DataContext` Refactor. Migration of datasource and store (5404)
* [MAINTENANCE] Add unit and integration tests for Splitting on Multi-Column Values (5464)
* [MAINTENANCE] Refactor `DataContextVariables` to leverage `property` and `setter` (5446)
* [MAINTENANCE] expect_profile_numeric_columns_diff_between_threshold_range (5467) (thanks stevensecreti)
* [MAINTENANCE] Make `DataAssistantResult` fixtures module scoped (5465)
* [MAINTENANCE] Remove keyword arguments within table row count expectations (4874) (thanks andyjessen)
* [MAINTENANCE] Add unit tests for Splitting on Converted DateTime (5470)
* [MAINTENANCE] Rearrange integration tests to insure categorization into proper deployment-style based lists (5471)
* [MAINTENANCE] Provide better error messaging if batch_request is not supplied to DataAssistant.run() (5473)
* [MAINTENANCE] Adds run time envvar for Snowflake Partner ID (5485)
* [MAINTENANCE] fixed algolia search page (5099)
* [MAINTENANCE] Remove pyspark<3.0.0 constraint for python 3.7 (5496)
* [MAINTENANCE] Ensure that `parter-integration` pipeline only runs on cronjob (5500)
* [MAINTENANCE] Adding fixtures Query Expectations tests (5486)
* [MAINTENANCE] Misc updates to `GeCloudStoreBackend` to better integrate with GE Cloud (5497)
* [MAINTENANCE] Update automated release schedule (5488)
* [MAINTENANCE] Update core-team in `teams.yml` (5489)
* [MAINTENANCE] Update how_to_create_a_new_expectation_suite_using_rule_based_profile… (5495)
* [MAINTENANCE] Remove pypandoc pin in constraints-dev.txt. (5501)
* [MAINTENANCE] Ensure that `add_datasource` method on `AbstractDataContext` does not persist by default (5482)