Great-expectations

* [ENHANCEMENT] Improve support for quantiles calculation in Athena
* [ENHANCEMENT] V3 API CLI docs commands have better error messages and more consistent short flags
* [ENHANCEMENT] Update all Data Connectors to allow for `batch_spec_passthrough` in config
* [ENHANCEMENT] Update `DataConnector.build_batch_spec` to use `batch_spec_passthrough` in config
* [ENHANCEMENT] Update `ConfiguredAssetSqlDataConnector.build_batch_spec` and `ConfiguredAssetFilePathDataConnector.build_batch_spec` to properly process `Asset.batch_spec_passthrough`
* [ENHANCEMENT] Update `SqlAlchemyExecutionEngine.get_batch_data_and_markers` to handle `create_temp_table` in `RuntimeQueryBatchSpec`
* [ENHANCEMENT] Usage stats messages for the v3 API CLI are now sent before and after the command runs 2661
* [ENHANCEMENT} Update the datasource new notebook for improved data asset inference
* [ENHANCEMENT] Update the `datasource new` notebook for improved data asset inference
* [ENHANCEMENT] Made stylistic improvements to the `checkpoint new` notebook
* [ENHANCEMENT] Add mode prompt to suite new and suite edit 2706
* [ENHANCEMENT] Update build_gallery.py script to better-handle user-submitted Expectations failing 2705
* [ENHANCEMENT] Docs + Tests for passing in reader_options to Spark 2670
* [ENHANCEMENT] Adding progressbar to validator loop 2620 (Thanks peterdhansen!)
* [ENHANCEMENT] Great Expectations Compatibility with SqlAlchemy 1.4 2641
* [ENHANCEMENT] Athena expect column quantile values to be between 2544 (Thanks RicardoPedrotti!)
* [BUGFIX] Rename assets in SqlDataConnectors to be consistent with other DataConnectors 2665
* [BUGFIX] V3 API CLI docs build now opens all built sites rather than only the last one
* [BUGFIX] Handle limit for oracle with rownum 2691 (Thanks NathanFarmer!)
* [BUGFIX] add create table logic for athena 2668 (Thanks kj-9!)
* [BUGFIX] Add note for user-submitted Expectation that is not compatible with SqlAlchemy 1.4 (uszipcode) 2677
* [BUGFIX] Usage stats cli payload schema 2680
* [BUGFIX] Rename assets in SqlDataConnectors 2665
* [DOCS] Update how_to_create_a_new_checkpoint.rst with description of new CLI functionality
* [DOCS] Update Configuring Datasources documentation for V3 API CLI
* [DOCS] Update Configuring Data Docs documentation for V3 API CLI
* [DOCS] Update Configuring metadata stores documentation for V3 API CLI
* [DOCS] Update How to configure a Pandas/S3 Datasource for V3 API CLI
* [DOCS] Fix typos in "How to load a database table, view, or query result as a batch" guide and update with `create_temp_table` info
* [DOCS] Update "How to add a Validation Operator" guide to make it clear it is only for V2 API
* [DOCS] Update Version Migration Guide to recommend using V3 without caveats
* [DOCS] Formatting fixes for datasource docs 2686
* [DOCS] Add note about v3 API to How to use the Great Expectations command line interface (CLI) 2675
* [DOCS] CLI SUITE Documentation for V3 2687
* [DOCS] how to share data docs on azure 2589 (Thanks benoitLebreton-perso!)
* [DOCS] Fix typo in Core concepts/Key Ideas section 2660 (Thanks svenhofstede!)
* [DOCS] typo in datasource documentation 2654 (Thanks Gfeuillen!)
* [DOCS] fix grammar 2579 (Thanks carlsonp!)
* [DOCS] Typo fix in Core Concepts/ Key Ideas section 2644 (Thanks TremaMiguel!)
* [DOCS] Corrects wrong pypi package in Contrib Packages README 2653 (Thanks mielvds!)
* [DOCS] Update dividing_data_assets_into_batches.rst 2651 (Thanks lhayhurst!)
* [MAINTENANCE] Temporarily pin sqlalchemy (1.4.9) and add new CI stage 2708
* [MAINTENANCE] Run CLI tests as a separate stage in Azure pipelines 2672
* [MAINTENANCE] Updates to usage stats messages & tests for new CLI 2689
* [MAINTENANCE] Making user configurable profile test more robust; minor cleanup 2685
* [MAINTENANCE] remove cli.project.upgrade event 2682
* [MAINTENANCE] column reflection fallback should introspect one table (not all tables) 2657 (Thank you peterdhansen!)
* [MAINTENANCE] Refactor Tests to Use Common Libraries 2663

- [BREAKING-EXPERIMENTAL] The `batch_data` attribute of `BatchRequest` has been removed. To pass in in-memory dataframes at runtime, the new `RuntimeDataConnector` should be used
- [BREAKING-EXPERIMENTAL] `RuntimeDataConnector` must now be passed Batch Requests of type `RuntimeBatchRequest`
- [BREAKING-EXPERIMENTAL] The `PartitionDefinitionSubset` class has been removed - the parent class `IDDict` is used in its place
- [BREAKING-EXPERIMENTAL] `partition_request` was renamed `data_connector_query`. The related `PartitionRequest` class has been removed - the parent class `IDDict` is used in its place
- [BREAKING-EXPERIMENTAL] `partition_definition` was renamed `batch_identifiers`. The related `PartitionDefinition` class has been removed - the parent class `IDDict` is used in its place
- [BREAKING-EXPERIMENTAL] The `PartitionQuery` class has been renamed to `BatchFilter`
- [BREAKING-EXPERIMENTAL] The `batch_identifiers` key on `DataConnectorQuery` (formerly `PartitionRequest`) has been changed to `batch_filter_parameters`
- [ENHANCEMENT] Added a new `RuntimeBatchRequest` class, which can be used alongside `RuntimeDataConnector` to specify batches at runtime with either an in-memory dataframe, path (filesystem or s3), or sql query
- [ENHANCEMENT] Added a new `RuntimeQueryBatchSpec` class
- [BUGFIX] Fixed issue where Sorters were not being applied correctly when `data_connector_query` contained limit or index 2617
- [DOCS] Updated docs to reflect above class name changes
- [DOCS] Added the following docs: "How to configure sorting in Data Connectors", "How to configure a Runtime Data Connector", "How to create a Batch Request using an Active Data Connector", "How to load a database table, view, or query result as a Batch"
- [DOCS] Updated the V3 API section of the following docs: "How to load a Pandas DataFrame as a Batch", "How to load a Spark DataFrame as a Batch"

* [ENHANCEMENT] CLI `docs list` command implemented for v3 api 2612
* [MAINTENANCE] Add testing for overwrite_existing in sanitize_yaml_and_save_datasource 2613
* [ENHANCEMENT] CLI `docs build` command implemented for v3 api 2614
* [ENHANCEMENT] CLI `docs clean` command implemented for v3 api 2615
* [ENHANCEMENT] DataContext.clean_data_docs now raises helpful errors 2621
* [ENHANCEMENT] CLI `init` command implemented for v3 api 2626
* [ENHANCEMENT] CLI `store list` command implemented for v3 api 2627

* [FEATURE] Added support for references to secrets stores for AWS Secrets Manager, GCP Secret Manager and Azure Key Vault in `great_expectations.yml` project config file (Thanks Cedric-Magnan!)
* [ENHANCEMENT] Datasource CLI functionality for v3 api and global --assume-yes flag 2590
* [ENHANCEMENT] Update UserConfigurableProfiler to increase tolerance for mostly parameter of nullity expectations
* [ENHANCEMENT] Adding tqdm to Profiler (Thanks peterdhansen). New library in requirements.txt
* [ENHANCEMENT][MAINTENANCE] Use Metrics to Protect Against Wrong Column Names
* [BUGFIX] Remove parentheses call at os.curdir in data_context.py 2566 (thanks henriquejsfj)
* [BUGFIX] Sorter Configuration Added to DataConnectorConfig and DataConnectorConfigSchema 2572
* [BUGFIX] Remove autosave of Checkpoints in test_yaml_config and store SimpleCheckpoint as Checkpoint 2549
* [BUGFIX] Populate (data) asset name in data docs for SimpleSqlalchemy datasource (Thanks xaniasd)
* [BUGFIX] pandas partial read_ functions not being unwrapped (Thanks luke321321)
* [BUGFIX] Don't stop SparkContext when running in Databricks (2587) (Thanks jarandaf)

- [FEATURE] Allow custom data_asset_name for in-memory dataframes 2494
- [ENHANCEMENT] Restore cli functionality for legacy checkpoints 2511
- [BUGFIX] Can not create Azure Backend with TupleAzureBlobStoreBackend 2513 (thanks benoitLebreton-perso)
- [BUGFIX] force azure to set content_type='text/html' if the file is HTML 2539 (thanks benoitLebreton-perso)
- [BUGFIX] Temporarily pin SqlAlchemy to < 1.4.0 in requirements-dev-sqlalchemy.txt 2547
- [DOCS] Fix documentation links generated within template 2542 (thanks thejasraju)
- [MAINTENANCE] Remove deprecated automerge config 2492

* [ENHANCEMENT] Improve support for median calculation in Athena (Thanks kuhnen!) 2521
* [ENHANCEMENT] Update `suite scaffold` to work with the UserConfigurableProfiler 2519
* [MAINTENANCE] Add support for spark 3 based spark_config 2481