Cumulusci

Issues closed:

- Fixed a bug introduced in CumulusCI 3.20.0 in which the ``upload_beta`` and ``upload_production`` tasks could hit a connection error if uploading the package took over 10 minutes.
- We corrected edge cases in how we processed Boolean options for the ``custom_settings_wait``, ``exec_anon``, and ``uninstall_post`` tasks. (Thanks to davidjray)

Critical Changes:

* We’ve removed the standard flow: `retrieve_scratch`. The recommended way for retrieving source-tracked changes is to use the `retrieve_changes` task.
* Changes to automatic merging:
* The `github_master_to_feature` task has been renamed to `github_automerge_main`. It still merges changes from the default branch to feature branches. In the case of an orphaned feature branch (a branch with a name like `feature/parent__child` where `feature/parent` does not exist as its own branch), the `github_automerge_main` branch will no longer merge to the orphaned branch.
* The `github_parent_to_children` task has been renamed to `github_automerge_feature`. It still merges changes from feature branches to their children (e.g. `feature/parent` would be merged to `feature/parent__child`). It is now possible to use multiple double-underscores to create more deeply nested children, and the task will only merge to the next level (e.g. `feature/parent` would merge to `feature/parent__child` which would merge to `feature/parent__child__grandchild`).
* The `children_only` option for these tasks has been removed. The strategy for picking which branches to target for merging is now determined by the source_branch.

Tasks, Flows, and Automation:

* `cci flow list` now displays flows in different groups that are organized by functional area. (This is similar to how cci task list currently works).
* The `insert_record` task can now be used against the Tooling API. We clarified that this task can accept a dict of values if configured in `cumulusci.yml`.
* Added support for newer metadata types to the `update_package_xml` task.
* Previously, large data loads and extracts would use enormous amounts of memory. Now they should use roughly constant amounts of memory.
* Adjusted tasks: install_managed and update_dependencies can now install packages from just a version id (04t).
* Added support for creating 2GP packages (experimental)
* New task: `github_package_data` gets a package version id from a GitHub commit status. It is intended primarily for use as part of the `ci_feature_2gp` flow. Implementation details can be found in the [features](https://cumulusci.readthedocs.io/en/latest/features.html) section of the documentation.
* New task: `create_package_version` - Builds a 2gp package (managed or unlocked) via a Dev Hub org. Includes some automated handling of dependencies:
* New Flow: `build_feature_test_package` - Runs the `create_package_version` task, and in the context of MetaCI it will set a commit status with the package version id.
* New Flow: `ci_feature_2gp` - Retrieves the package version from the commit status set by build_feature_test_package, installs dependencies and the package itself in a scratch org, and runs Apex tests. (There is another new task, `github_package_data`, which is used by this flow.)

User Experience:

* Improved error messaging when encountering errors during bulk data mapping validation.

Issues Closed:

* Fixed a very rare bug that caused CumulusCI to fail to retrieve installed packages from an org when running package-related tasks or evaluating when conditional expressions.
* Fixed UnicodeDecodeError while opening config files on Windows.
* Fixed a bug in `cumulusci.core.sfdx.sfdx` when `capture_output` is False

Issues closed:

- Fixed an issue (2032) where REST API data loads incorrectly handled Boolean values.

Changes:

- Tasks and automation:

- CumulusCI now supports using the REST Collections API in data load, extract, and delete operations. By default, CumulusCI will select an API for you based on data volume (<2000 records uses the REST API, >=2000 uses Bulk); a desired API can be configured via the mapping file.
- Removed the namespace_tokenize option from tasks that deploy metadata, and removed the namespace_inject option from tasks that retrieve metadata, because it's unclear when they would be useful.
- The task create_permission_set allows for creating and assigning a Permission Set that enables specific User Permissions. (Note: other types of permissions are not yet supported).
- The task create_bulk_data_permission_set creates a Permission Set with the Hard Delete and Set Audit Fields permissions for use with data load operations. The org permission to allow Set Audit Fields must be turned on.
- CumulusCI's load_dataset and extract_dataset tasks now support relative dates. To take advantage of relative dates, include the anchor_date key (with a date in YYYY-MM-DD format) in each mapping step you wish to relativize. On extract, dates will be modified to be the same interval from the anchor date as they are from the current date; on load, dates will be modified to be the same interval from today's date as they are from their anchor. Both date and date-time fields are supported.

Changes:

- Tasks and automation:
- CumulusCI now tries 10 times (instead of 5) to install managed
package versions, which helps ameliorate timeouts when new
versions are released.
- We added support for CSV files to the `push_list` task.
- We added a `ref` option to `github_copy_subtree` to allow
publishing a git reference (commit hash, branch, or tag).
- Changed the `disable_tdtm_trigger_handlers`
(SetTDTMHandlerStatus) task so that trigger handler state is
remembered in the cache directory instead of `REPO_ROOT`.
- User experience:
- The `cci error info` command now defaults to showing the entire
traceback when it is more than 30 lines.
- Robot Framework:
- The following robot keywords have been updated to work with
Winter '21:
- `Load related list`
- `Click related list button`
- `Click related item link`
- `Click related item popup link`
- `Go to object home`
- `Go to object list`
- `Go to record home`
- `Populate lookup field`
- The keyword `Load related list` has been rewritten to be
slightly more efficient. It also has a new parameter `tries`
which can be used if the target is more than 1000 pixels below
the bottom of the window.

Issues Closed:

- Fixed a bug where `cci error gist` could throw a UnicodeDecodeError
on Windows (fixes \1977)
- Fixed a bug where `cci org list` could throw a TypeError when run
outside a project directory (fixes \1998)
- The `metadeploy_publish` task can now update translations for
language codes with more than 2 letters.
- Fixed a bug where the `extract_dataset` task could fail with a
UnicodeEnodeError on Windows.
- `update_dependencies` deduplicates its package install list, making
it possible to handle situations where the same beta package is
reached by two dependency paths.

Changes:

- Tasks and automation:

- We added the `upload_user_profile_photo` and `upload_default_user_profile_photo` tasks, which allow for setting Users' profile photos from images stored in the repository. (Thanks to spelak-salesforce).
- We added the property `is_person_accounts_enabled` to the `org_config` object, which is available in `when` clauses. (Thanks to spelak-salesforce).

- Policies and inclusive language:

- We added information about Salesforce's Open Source Community Code of Conduct and Security policies.
- We updated documentation to more consistently refer to the "main" branch, reflecting CumulusCI's support for per-project specification of main branches other than `master`.

- User experience:

- We modified how we handle situations where the default org is not valid for better user experience.
- We catch a common mistake in entering command-line options (`-org` instead of `--org`, as well as incorrectly-formatted flow options) and show a clearer error.
- We now capture and display the `InstanceName` of orgs in `cci org list`'s output.

- Robot Framework:

- We now cleanly fall back to the latest available API version for Robot locators if the newest API version does not have an available locator file. This change helps support Robot testing on the latest prerelease editions of Salesforce.
- We included some updates to locators for API version 50.0.

- Other:

- We added a new environment variable, `SFDX_SIGNUP_INSTANCE`, and an `instance` key in org definitions, to specify a preferred instance routing. NOTE: this functionality requires Dev Hub permissions that are not Generally Available.

Issues closed:

- Fixed a bug which prevented package install links from getting added to release notes.
- Fixed a bug (1914) which caused errors when customizing some Flow steps with decimal step numbers.
- Fixed a bug making it difficult to troubleshoot issues with Snowfakery and CumulusCI on Windows.
- Fixed a bug in `update_admin_profile` that resulted in errors when attempting to manage Record Types across multiple objects.