Great-expectations

* (BREAKING) Clarified API language: renamed all ``generator`` parameters and methods to the more correct
``batch_kwargs_generator`` language. Existing projects may require simple migration steps. See :ref:`Upgrading to 0.10.x` for instructions.
* Adds anonymized usage statistics to Great Expectations. You can opt out at any time, but we’re hoping that you
won’t: this data will be very helpful for improving the product. See this article for details: :ref:`Usage Statistics`.
* CLI: improve look/consistency of `docs list`, `suite list`, and `datasource list` output; add `store list` and `validation-operator list` commands great_expectations/core/logging/usage_statistics.py
* New SuiteBuilderProfiler that facilitates faster suite generation by allowing columns to be profiled
* Added two convenience methods to ExpectationSuite: get_table_expectations & get_column_expectations
* Added optional profiler_configuration to DataContext.profile() and DataAsset.profile()
* Added list_available_expectation_types() to DataAsset

What’s Changed
* Data Docs: improve configurability of site_section_builders
* TupleFilesystemStoreBackend now ignore `.ipynb_checkpoints` directories `1203 <https://github.com/great-expectations/great_expectations/issues/1203>`_
* Data docs builder init (1275) roblim
* Remove profile CLI command parameter docstring (1272) mzjp2

What’s Changed
-----------------
* Allow evaluation parameters support in run_validation_operator
* Add log_level parameter to jupyter_ux.setup_notebook_logging.
* Add experimental display_profiled_column_evrs_as_section and display_column_evrs_as_section methods, with a minor (nonbreaking) refactor to create a new _render_for_jupyter method.
* Allow selection of site in UpdateDataDocsAction with new arg target_site_names in great_expectations.yml
* Fix issue with regular expression support in BigQuery (1244)
* Allow UpdateDataDocsAction to also specify data doc sites to build (1256) troxil
* Fix indentation in create_expectations page (1260) ian-whitestone
* Fix spelling in data docs modal (1254) Bussmeyer

What’s Changed

* Allow basic operations in evaluation parameters, with or without evaluation parameters.
* When unexpected exceptions occur (e.g., during data docs rendering), the user will see detailed error messages, providing information about the specific issue as well as the stack trace.
* Remove the "project new" option from the command line (since it is not implemented; users can only run "init" to create a new project).
* Update type detection for bigquery based on driver changes in pybigquery driver 0.4.14. Added a warning for users who are running an older pybigquery driver
* added execution tests to the NotebookRenderer to mitigate codegen risks
* Add option "persist", true by default, for SparkDFDataset to persist the DataFrame it is passed. This addresses 1133
in a deeper way (thanks tejsvirai for the robust debugging support and reproduction on spark).
- Disabling this option should *only* be done if the user has *already* externally persisted the DataFrame, or if the
dataset is too large to persist but *computations are guaranteed to be stable across jobs*.
* Enable passing dataset kwargs through datasource via dataset_options batch_kwarg.
* Fix AttributeError when validating expectations from a JSON file
* Data Docs: fix bug that was causing erratic scrolling behavior when table of contents contains many columns
* Make expectation suite json deterministically human-parseable (1247) mzjp2
* Change the verb of the cli command project in docs (1237) Bussmeyer
* Fix AttributeError when validating expectations from a JSON file (1238) eorbe
* Fixed link to typical_workflow (1227) jvanbuel
* Remove Python 2 tags from setup.py (1229) mzjp2

Changes

* Update marshmallow dependency to >3. NOTE: as of this release, you MUST use marshamllow >3.0, which REQUIRES python 3. 1187 jcampbell
- Schema checking is now stricter for expectation suites, and data_asset_name must not be present as a top-level
key in expectation suite json. It is safe to remove.
- Similarly, datasource configuration must now adhere strictly to the required schema, including having any
required credentials stored in the "credentials" dictionary.
* New beta CLI command: `tap new` that generates an executable python file to expedite deployments. 1193 Aylr
* Added feature maturity in README 1203 kyleaton

Fixes

* Fix failing test that should skip if postgresql not running 1199 cicdw
* bugfix in TableBatchKwargsGenerator docs

* validate result dict when instantiating an ExpectationValidationResult (1133)
* DataDocs: Expectation Suite name on Validation Result pages now link to Expectation Suite page
* `great_expectations init`: cli now asks user if csv has header when adding a Spark Datasource with csv file
* Improve support for using GCP Storage Bucket as a Data Docs Site backend (thanks hammadzz)
* fix notebook renderer handling for expectations with no column kwarg and table not in their name (1194 )