Cumulusci

Breaking changes

* We reorganized the flows for setting up a package for regression testing for better symmetry with other flows. If you were running the ``install_regression`` flow before, you now probably want ``regression_org``.

Details: The ``install_regression`` flow now installs the package _without_ configuring it.
There is a new ``config_regression`` flow to configure the package (it defaults to calling ``config_managed``)
and a ``regression_org`` flow that includes both ``install_regression`` and ``config_regression``.

New features

* CumulusCI now has experimental support for deploying projects in [DX source format](https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_source_file_format.htm).
To enable this, set ``source_format: sfdx`` in the project section of ``cumulusci.yml``.
CumulusCI will deploy DX-format projects to scratch orgs using ``sfdx force:source:push`` and to other orgs using the Metadata API (by converting to metadata source format in a temporary directory).
* Setting a default org in CumulusCI (using ``cci org default`` or the ``--default`` flag when creating a scratch org) will now also update the sfdx ``defaultusername``. (868)
* When connecting to GitHub using ``cci service connect github``, CumulusCI will now check to make sure the credentials are valid before saving them.
* Robot Framework:

* Added a framework for creating "page object" classes to contain keywords related to a particular page or component.
* The ``robot`` task now takes a ``name`` option to control the name of the robot suite in output.
* Updates to the keyword ``Open Test Browser``:

* It allows you to open more than one browser in a single test case. (1068)
* It sets the default size for the browser window to 1280x1024.
* Added a new keyword argument ``size`` to override the default size.
* Added a new keyword argument ``alias`` to let you assign an alias to multiple browser windows.

Issues fixed

* Robot Framework: Fixed a bug where the ``Delete Session Records`` keyword would skip deleting some records. (973)
* If Salesforce returns an error response while refreshing an OAuth token, CumulusCI will now show the response instead of just the HTTP status code.
* Fixed a bug in reporting errors from the Metadata API if the response contains ``componentFailures`` with no ``problem`` or ``problemType``.

New features:

* Added tasks ``list_changes`` and ``retrieve_changes`` which interact with source tracking in scratch orgs to handle retrieving changed metadata as Metadata API format source.
* Added task ``EnsureRecordTypes`` to generate a Record Type and optional Business Process for a specific sObject and deploy the metadata, if the object does not already have Record Types.
* The ``update_admin_profile`` task now uses Python string formatting on the ``package.xml`` file used for retrieve. This allows injection of namespace prefixes using ``{managed}`` and ``{namespaced_org}``.

Issues fixed:

* If CumulusCI gets a connection error while trying to call the Salesforce Metadata API, it will now retry several times before giving up.
* The GitHub release notes parser now recognizes Issues Closed if they are linked in Markdown format.
* Robot Framework: Fixed a locator used by the ``Select App Launcher App`` keyword to work in Summer '19.
* The ``cci project init`` command now uses an updated repository URL when extending EDA.

* Allow configuration of the email address assigned to scratch org users, with the order of priority being (1) any ``adminEmail`` key in the scratch org definition; (2) the ``email_address`` property on the scratch org configuration in ``cumulusci.yml``; (3) the ``user.email`` configuration property in Git.
* CumulusCI can now handle building static resource bundles (``*.resource``) while deploying using the Metadata API. To use this option, specify the ``static_resource_path`` option for the deploy task. Any subdirectory in this path will be turned into a resource file and added to the package during deployment. There must be a corresponding ``*.resource-meta.xml`` file for each static resource bundle.
* Bulk data tasks: Fixed a bug that added extra underscores to field names when processing lookups.
* Robot Framework: The Salesforce library now has the ability to switch between different sets of locators based on the Salesforce version, and thanks to it we've fixed the robot so it can click on modal buttons in the Summer '19 release.
* The ``cci project init`` command now generates projects with a different preferred structure for Robot Framework tests and resources, with everything inside the ``robot`` directory. Existing projects with tests in the ``tests`` directory should continue to work.

* The ``purgeOnDelete`` flag for the ``deploy`` task will now automatically be set to false when
deploying metadata to production orgs (previously deployment would fail on production orgs
if this flag was true).
* The installation documentation now recommends using ``pipx`` to install CumulusCI on Windows,
so that you don't have to set up a virtualenv manually.

Changes:

* Updated the default Salesforce Metadata API version to 45.0
* The scratch org definition files generated by ``cci project init`` now use ``orgPreferenceSettings`` instead of the deprecated ``orgPreferences``.
* The ``metadeploy_publish`` task now defaults to describing tasks based on ``Deploy`` as "metadata" steps instead of "other".

Issues Fixed:

* Fixed a couple problems with generating passwords for new scratch orgs:

* A project's predefined scratch org configs now default to ``set_password: True`` (which was already the case for orgs created explicitly using cci org scratch).
* A scratch org config's ``set_password`` flag is now retained when recreating an expired org. (Fixes 670)

* Fixed the logic for finding the most recent GitHub release so that it now only considers tags that start with the project's git ``prefix_release``.
* Fixed the ``install_prod_no_config`` flow. The ``deploy_post`` task was not injecting namespace tokens correctly.
* Fixed the ``connected_app`` task to work with version 7 of the sfdx CLI. (Fixes 1013)
* Robot Framework: Fixed the ``Populate Field`` keyword to work around intermittent problems clearing existing field values.

Changes:

* Tasks can now be placed in groups for the task list! Just specify a ``group`` when defining the task in YAML.

* By popular request, there is now an ``org import`` command to import an org from the SFDX keychain to the CumulusCI keychain. It takes two arguments: the SFDX username or alias, and the org name.

* Robot Framework:

* The ``Populate Field`` keyword now clears an existing value using keystrokes to make sure that change events are fired.
* Added a ``Get Namespace Prefix`` keyword to the CumulusCI library to get the namespace prefix for a package.
* Fixed a bug that broke opening a browser after using the ``Run Task`` keyword.

* Documentation updates:

* The readme now includes a link to the full documentation.
* The instructions for installing CumulusCI on macOS have been simplified and now recommend using the official Python installer from python.org instead of Homebrew. (Homebrew should still work fine, but is no longer necessary.) We also now suggest creating a virtualenv using venv rather than pyenv since the former is included with Python. It's fine to continue using pyenv if you want.
* Give more useful links for how to set up SFDX.
* Updated robot library docs.

* Internal refactoring:

* Removed dependency on HiYaPyCo for YAML loading, which would not report which file failed to load in the event of a YAML parse error.
* We now consistently load YAML in the same manner throughout the entire library, which will work with all supported Python versions.
* Simplified the Python API for setting up a CumulusCI runtime. Begone, YamlGlobalConfig and YamlProjectConfig. Our Python API is not yet documented, but we're working on it. In the meantime, if you were relying on running CCI from within Python, you can now just use BaseGlobalConfig (and its get_project_config member) to bootstrap CCI.
* BaseProjectConfig has shrugged off some methods that just delegated to the keychain.
* BaseGlobalConfig has shrugged off some unimplemented methods, and BaseGlobalConfig.get_project_config is now deprecated in favor of using a runtime.
* Introducing... 🥁CumulusCIRuntime! In order to alleviate the complexities of getting CumulusCI tasks/flows running from within a Python application, CumulusCIRuntime encapsulates a lot of the details and wiring between Keychain, GlobalConfig, and ProjectConfig. Usage docs are barely included.
* CliConfig has been renamed to CliRuntime and now inherits from CumulusCIRuntime. It is still accessible as CliConfig.
* Upgraded dependencies.

* Contributor improvement: The contributor docs now explain how to install pre-commit hooks to make sure our linters have run before you commit.

Issues Closed:

* 674: ``cci org import <username> <org_name>``
* 877: CumulusCI should be able to connect to any DX alias and/or understand dx auth files