Cumulusci

Changes:

- A new <span class="title-ref">snowfakery</span> task with better usability and multi-processor support. Look at the CumulusCI docs to learn the new syntax: <https://cumulusci.readthedocs.io/en/stable/data.html#generate-fake-data> (2705)
- CumulusCI now uses Snowfakery 2.0, with various new features, especially the ability to query into orgs. More information: <https://github.com/SFDO-Tooling/Snowfakery/releases/tag/2.0> (#2705)
- We improved our Robot documentation so that it's possible to link to keyword documentation instead of having to download it locally (2696)
- CumulusCI uses a new port (7788) for the built-in connected app to lessen the chances that the port is in use. (2698)
- CumulusCI now checks if the port associated with a callback URL/redirect URI is in use during OAuth2 flows, and if so, raises a more friendly error. (2698)
- The `generate_data_dictionary` task now includes Custom Settings, Custom Metadata Types, and Platform Events. (2712)
- The `generate_data_dictionary` task now excludes any schema with visibility set to Protected. This behavior can be turned off (including protected schema) with the `include_protected_schema` option. (2712)
- The `generate_data_dictionary` task now parses object and field metadata anywhere in a Salesforce DX release other than in the `unpackaged/` directory tree. (2712)
- Builds that install feature-test 2GP packages now present a cleaner error message when the current commit is not found on GitHub. (2713)
- SFDX and CumulusCI both support noancestors as a Scratch org config option but CumulusCI generated a warning if users tried to specify the option in cumulusci.yml. (2721)

Issues closed:

- Fixed issue where CumulusCI did not correctly convert a package version specified as a number in YAML to a string. This now raises a warning. (2692)
- Fixed a bug where OAuth errors were not reported in detail. (2694)
- Fixed an issue where CumulusCI did not grant permissions to Custom Tabs when running `update_admin_profile` without a custom `package.xml`. Projects that use a custom `package.xml` with `update_admin_profile` should update their manifest to include a `CustomTab` wildcard for the same outcome. (2699)
- Fixed an issue where the `dx`, `dx_push`, and `dx_pull` tasks did not refresh the org's access token. (2703)
- Fixed issues in the `generate_data_dictionary` task that resulted in failures when processing fields with blank Help Text or processing standard fields. (2706)
- Fixed an issue preventing `generate_data_dictionary` from working with four-digit (1.0.0.0) 2GP version numbers. (2712)
- Fixed an issue causing `release_2gp_beta` to fail to create a GitHub release with a dependency-parsing error. (2720)

Changes:

* The built-in connected app that CumulusCI uses by default is now visible in the output of the ``cci service list`` command. This makes it possible to switch back and forth between this connected app and another one as the current default when multiple connected_app services are configured. The built-in connected_app service has the name ``built-in`` and cannot be renamed or removed. (2664)
* The ``generate_data_dictionary`` task includes a new option, ``include_prerelease``. If set to ``True``, CumulusCI will include unreleased schema in the data dictionary from the current branch on GitHub, with the version listed as "Prerelease". (2671)
* Added a new task, ``gather_release_notes``, which generates an HTML file with release notes from multiple repositories. (2633)
* The ``deploy_marketing_cloud_package`` task includes a new option, ``custom_inputs``, which can be used to specify values to fill in for inputs in a Marketing Cloud package. (2683)
* Mappings for the ``extract_dataset`` task can now specify a ``soql_filter`` to restrict which records are extracted. Thanks sfdcale (2663)
* Robot Framework: The ``Scroll Element Into View`` keyword in the Salesforce library now scrolls the center of the element into view rather than the top. (2689)

Issues closed:

* Fixed a bug where CumulusCI could not parse the repository owner and name from an ssh git remote URL if it used an ssh alias instead of ``github.com``. (2684)
* Fixed a bug where ``cci service info <service_type>`` would display ``None`` as the name for the default service if no name was provided. (2664)
* Fixed a missing dependency on the ``contextvars`` Python package in Python 3.6.

Changes

- The `install_managed` task now supports 2GP releases (2655).
- We changed the behavior of the `release_2gp_beta` flow to always upload a package version, even if metadata has not changed (2651).
- We now support sourcing install keys for packages from environment variables via the `password_env_name` dependency key (2622).

Robot Framework

- We upgraded SeleniumLibrary to 5.x (2660).
- We added a new keyword "select window" to Salesforce library, to replace the keyword of the same name which was renamed in SeleniumLibrary 5.x to 'switch window'. We will be removing this keyword in a future release; tests should use 'switch window' instead.

Issues Closed

- We corrected some JavaScript issues that were occurring with Chrome 91. (2652)
- We fixed a bug impacting the `generate_data_dictionary` task when used with dependencies (2653).
- We fixed an issue causing `sfdx` commands that had options with spaces to fail to execute on Windows (2656).
- We fixed an issue causing the creation of incorrect 2GP beta tags (2651).

Changes

- Added the option `tag_prefix` to the `github_release` task. This option can be set to specify what prefix you would like to use when CumulusCI creates a release tag for you in GitHub. (\2642)
- The `deploy_marketing_cloud_package` task has been updated to match changes to the Marketing Cloud Package Manager API. It also now raises an exception if the deployment failed. (\2632)

Robot Framework

- Improved the output of the `robot_libdoc` task. (\2627)

Data generation with Snowfakery

- Updated to [Snowfakery 1.12](https://github.com/SFDO-Tooling/Snowfakery/releases/tag/v1.12) (\#2538)

Issues Closed

- Fixed an issue where flow reference documentation was rendering with an error. (\2646)
- CumulusCI will now remove orgs when the `--delete-org` option is passed to `cci flow run`, even if an error occurs while running the flow. (\2644)
- Fixed a bug where beta tags created via the `release_2gp_beta` flow were not receiving the proper tag prefix. (\2642)
- Fixed namespace injection for filenames with a `___NAMESPACE___` token in sfdx format. (\2631) (Thanks bethbrains)
- Fixed a bug in `cci org connect` where the `--sandbox` flag was directing users to login at `login.salesforce.com` instead of `test.salesforce.com`. (\2630)
- Fixed a regression where the `skip` key for a dependency could no longer be specified as a single string instead of a list. (\2629)
- Fixed a regression in freezing the `deploy_pre`/`deploy_post` tasks for MetaDeploy install plans. (\2626)
- Fixed bugs in the `deploy_marketing_cloud_package` task's payload construction. (\2620, \2621)

Critical Changes

- Upgraded Robot Framework to 4.x. For information about new features
and some backward incompatibilities see the [Robot Framework 4.0
release
notes](https://github.com/robotframework/robotframework/blob/master/doc/releasenotes/rf-4.0.rst).
(\2603)
- The `update_dependencies` task now guarantees to resolve unpackaged
metadata directories (subdirectories of `unpackaged/pre` and
`unpackaged/post`) in alphabetical order, matching the behavior of
`deploy_pre` and `deploy_post`. `unpackaged/pre/bar` will deploy
prior to `unpackaged/pre/foo`. The previous behavior was undefined,
which caused rare problems. This change is critical only for
projects that have deployment-order dependencies between unpackaged
directories located in upstream dependencies and rely on the current
undefined load order. (\2588)

Changes

- The CumulusCI documentation has a new section: [Testing with
Second-Generation
Packaging](https://cumulusci.readthedocs.io/en/latest/2gp_testing.html)
(\2597)
- CumulusCI has two new service types: `oauth2_client` &
`marketing_cloud`. These are considered experimental. (\2602)
- The `marketing_cloud` service allows users to connect to a Marketing
Cloud tenant via OAuth so that tasks that work with Marketing Cloud
can make API calls on the user's behalf. (\2602)
- The `oauth2_client` service takes information for an individual
OAuth2 client which can then be used in place of the default client.
This currently applies only to the `marketing_cloud` service. To
setup a Marketing Cloud service with a specific OAuth2 client use:
`cci service connect marketing-cloud <name-of-service> --oauth_client <name-of-oauth-client>`.
(\2602)
- CumulusCI has a new task: `deploy_marketing_cloud_package`. This
task allows a user to pass the path to a .zip file to a Marketing
Cloud package (downloaded from the Marketing Cloud Package Manager)
and deploy the package via a `marketing_cloud` service (see above).
Note that successfully deploying a package using this task may
require permissions that are not generally available. (\2602)
- The `install_managed` and `install_managed_beta` tasks now take no
action if the specified package is already installed in the target
org. (\2590)
- The `cci org list` command can now output in `JSON` format by
passing it the `--json` flag. (\2593)

Issues Closed

- Fixed an issue parsing `cumulusci.yml` files that contained Unicode
characters on Windows. (\2617)
- Fixed an issue in the `github_copy_subtree` task where CumulusCI
would silently generate incorrect or truncated commits when a
directory was passed to the `include` task option. (\2601)
- The `deploy_pre` and `deploy_post` tasks avoid warnings by freezing
installer steps that match current expectations. (\2589)

We fixed a regression in the ``load_dataset`` task where some sObjects could not be loaded without explicitly turning off the new ``set_recently_viewed`` option.