Nextstrain-cli

_These release notes are automatically extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme

Improvements

- Several kinds of errors from `nextstrain login` and `nextstrain whoami` related to their interactions with a remote server are now clearer. ([347](https://github.com/nextstrain/cli/pull/347))

_These release notes are automatically extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme


This release drops support for Python versions 3.6 and 3.7 and adds support for 3.11. ([325](https://github.com/nextstrain/cli/pull/325), [#327](https://github.com/nextstrain/cli/pull/327))

Note that this Python version support only matters if you're installing Nextstrain CLI from PyPI or Bioconda ([c.f.](https://docs.nextstrain.org/projects/cli/en/8.0.0/installation/)). It does not apply if you're installing Nextstrain CLI using the standalone installation method we recommend in the [Nextstrain installation documentation](https://docs.nextstrain.org/page/install.html). In that case, a supported Python version is always bundled with `nextstrain`.

Improvements

* The `nextstrain remote` family of commands now support alternative nextstrain.org-like remotes such as internal Nextstrain Groups Server instances and development instances of nextstrain.org. Authentication with these remotes is supported via `nextstrain login` and related commands. Remotes maintain their authentication alongside each other, e.g. you can be logged into nextstrain.org as well as an alternative nextstrain.org-like instance.

As an example, a Nextstrain Groups Server instance accessible at nextstrain.example.com could now be logged into and interacted with like so:

nextstrain login nextstrain.example.com
nextstrain whoami nextstrain.example.com
nextstrain remote ls nextstrain.example.com/groups/bedford-lab/

The default remote is still nextstrain.org. ([333](https://github.com/nextstrain/cli/pull/333))

* `nextstrain login` now performs authentication via a web browser by default (using OpenID Connect 1.0 and OAuth 2.0). The previously method of direct password entry is still used when a username is provided (e.g. with `--username` or `-u`). See `nextstrain login --help` for more information. ([333](https://github.com/nextstrain/cli/pull/333))

* With the new support for being logged into multiple remotes, `nextstrain logout` now also supports an `--all` flag to remove all locally-saved credentials without having to logout of each remote separately. ([333](https://github.com/nextstrain/cli/pull/333))

* `nextstrain remote upload` now skips gzip compression when uploading zstandard-compressed (`.zst`) files, matching its behaviour for other types
of compressed files. ([330](https://github.com/nextstrain/cli/pull/330))

* Commands that may automatically open a web browser, e.g. `nextstrain view` or `nextstrain login`, now respect the semi-conventional `NOBROWSER` environment variable to disable opening a browser. Set `NOBROWSER` to a non-empty value to disable automatic opening. When disabled, the URL to manually open will be shown. ([332](https://github.com/nextstrain/cli/pull/332))

* The error message emitted by the `nextstrain remote` family of commands when they're unable to access an S3 bucket now lists a few possible reasons for the failure. ([341](https://github.com/nextstrain/cli/pull/341))

* `nextstrain remote download` now supports downloading core datasets which are only visible on the web by direct access via their URL. For example, the following now work (where in previous versions they did not):

nextstrain remote download https://nextstrain.org/nextclade/sars-cov-2/21L
nextstrain remote download https://nextstrain.org/enterovirus/d68/vp1/2020-01-23

This support also covers [past snapshots of datasets](https://docs.nextstrain.org/en/latest/guides/versions.html), which is a recently-added feature to nextstrain.org. For example:

nextstrain remote download https://nextstrain.org/flu/seasonal/h3n2/ha/6y2023-07-01
nextstrain remote download https://nextstrain.org/ncov/gisaid/21L/global/6m2024-01-09

([345](https://github.com/nextstrain/cli/pull/345))

Bug fixes

* Commands making use of user authentication (e.g. `nextstrain login`, `nextstrain whoami`, some invocations of `nextstrain remote`, etc.) no longer fail if the logged in user is a member of no groups. ([323](https://github.com/nextstrain/cli/pull/323))

* On Linux and macOS, the standalone installer's final instructions now correctly suggest modifying `~/.profile` instead of, incorrectly, `~/.shrc` when the default shell is sh. ([321](https://github.com/nextstrain/cli/pull/321))

* The Singularity runtime once again supports Apptainer's `singularity` executable. The minimum version checking added in 7.0.0 broke usage of the Singularity runtime with Apptainer (compared with SingularityCE). Our intent is to support both lineages of Singularity. Thanks to osageorange for raising this issue and testing the fix! ([343](https://github.com/nextstrain/cli/pull/343))

_These release notes are automatically extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme


Improvements

* When attached to an AWS Batch job, `nextstrain build` now only requires confirmation to stop the job (e.g. a double Control-C/`SIGINT`) when stdin is a terminal (TTY). When stdin is not a terminal, as is frequently the case in automated or programmatic contexts, then `SIGINT` requests job cancellation immediately. ([308][])

* When attached to an AWS Batch job, `nextstrain build` now also detaches upon receiving `SIGHUP` (in addition to detaching upon receiving Control-Z/`SIGTSTP`). `SIGHUP` is sent, for example, when a user closes (or loses) the terminal in which `build` is still running. ([308][])

* When launching or attaching to an AWS Batch job, `nextstrain build` now supports the `--detach-on-interrupt` option to change Control-C/`SIGINT` to a request to detach from the job rather than to cancel it. This is useful in automated contexts where `SIGINT` may be sent outside of a user's control, and it's also handy as a molly-guard to avoid accidental cancellation when the intent is to just observe a job. ([308][])

* When attaching to an AWS Batch job, `nextstrain build` now supports the `--cancel` option to immediately request cancellation. ([308][])

[308]: https://github.com/nextstrain/cli/pull/308

Bug fixes

* We've plugged some isolation leaks in the Conda runtime where the [Python user site directory](https://docs.python.org/3/library/site.html), e.g. `~/.local/lib/pythonX.Y/site-packages`, as well as the [`PYTHONPATH` and `PYTHONHOME` environment variables](https://docs.python.org/3/using/cmdline.html#environment-variables) could influence and break the runtime. ([311](https://github.com/nextstrain/cli/pull/311))

_These release notes are extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme

_The changes below include those noted for 7.3.0, which was an unreleased version due to [test failures](https://github.com/nextstrain/cli/actions/runs/6238951959). All changes below were first released as part of 7.3.0.post1._

Improvements

* build: Providing a path to a pathogen build directory is no longer required when the AWS Batch runtime is in use (e.g. with `--aws-batch`) and both the `--attach` and `--no-download` options are given. This allows usages which just want to check job status or logs to stop providing a meaningless/unused directory. ([305](https://github.com/nextstrain/cli/pull/305))

* The initial setup of the Conda runtime, i.e. with `nextstrain setup conda`, now explicitly installs the latest available runtime package version. This was the implicit intent and expectation, but, for various Conda packaging reasons, it was (silently) not always the result. Now an error will occur if the latest version is not installable. This matches the existing behaviour when updating the Conda runtime, i.e. with `nextstrain update conda`. ([312](https://github.com/nextstrain/cli/pull/312))

Bug fixes

* The `HOST` and `PORT` environment variables are now respected by `nextstrain view`. They provide defaults when `--host` and/or `--port` aren't provided. ([310](https://github.com/nextstrain/cli/pull/310))

* Updated fsspec and s3fs dependencies to avoid version 2023.9.1, since they caused `nextstrain build --aws-batch` invocations to fail. ([313](https://github.com/nextstrain/cli/pull/313))

Development

* Update CI to test against the SingularityCE 3.x series only ([314](https://github.com/nextstrain/cli/pull/314))

_These release notes are automatically extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme


Bug fixes

* The Conda runtime now uses Micromamba 1.1.0 (upgraded from 1.0.0) to manage the runtime environment. The newer version has improved error handling in edge cases where the previous version silently failed when used during `nextstrain update`. ([301](https://github.com/nextstrain/cli/pull/301))

* The Singularity runtime now cleans up unnecessary caches during `nextstrain update` and `nextstrain setup`. This reduces the storage space required by the runtime and prevents it from growing unbounded over time. However, this also results in increases in network transfer during `nextstrain update`. We may mitigate the additional transfer in the future. ([303][])

Documentation

* The command-line `--help` output for commands and the corresponding documentation pages on the web are more integrated and improved in various small ways. In particular, command options are more cross-referencable and directly linkable and the wrap-width of `--help` output is more consistent and reliably readable. ([299](https://github.com/nextstrain/cli/pull/299))

* We've added a large section of new and expanded [documentation on runtimes](https://docs.nextstrain.org/projects/cli/page/runtimes/), both in general and for each individual runtime. ([#303][])

* We've added and expanded documentation about configuration, including the [config file](https://docs.nextstrain.org/projects/cli/page/config/file/) and the [paths](https://docs.nextstrain.org/projects/cli/page/config/paths/) used by Nextstrain CLI. Documentation in general now includes more about the environment variables that can be used to configure Nextstrain CLI. ([#303][])


[303]: https://github.com/nextstrain/cli/pull/303

_These release notes are automatically extracted from the full [changelog][]._

[changelog]: https://github.com/nextstrain/cli/blob/master/CHANGES.md#readme


Improvements

* Commands that use a runtime (`nextstrain build`, `nextstrain shell`, and `nextstrain view`) now support two new options for setting or passing thru environment variables into the runtime environment:

--env <name>[=<value>]
--envdir <path>

When either of these options are given, the default behaviour of automatically passing thru several "well-known" environment variables is disabled. That is, the following "well-known" environment variables are only automatically passed thru when the new `--env` and `--envdir` options are _not_ used:

- `AUGUR_RECURSION_LIMIT`
- `AUGUR_MINIFY_JSON`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN`
- `ID3C_URL`
- `ID3C_USERNAME`
- `ID3C_PASSWORD`
- `RETHINK_HOST`
- `RETHINK_AUTH_KEY`

Pass these variables explicitly via `--env` or `--envdir` if you need them in combination with other `--env` or `--envdir` usage. For more usage details, use the `--help-all` flag of any of those commands, e.g. `nextstrain build --help-all`. ([289](https://github.com/nextstrain/cli/pull/289))

* Environment variables are now passed to the Docker and AWS Batch runtimes via more secure means when the container image in use is new enough to support it (`nextstrain/base:build-20230613T204512Z` and newer). This ensures the env values aren't visible in the container's config (e.g. via `docker inspect`, `aws batch describe-jobs`, the AWS web console). If you're using an older image, you can update it with `nextstrain update docker`.

For Docker, environment variables are written to an internal and temporary envdir directory visible only to the current user which is deleted immediately after use at container start.

For AWS Batch, environment variables are written to a ZIP archive on S3, alongside but separate from the ZIP archive of the build dir. This env archive is deleted from S3 immediately after use at container start.

Both of these approaches minimize the amount of time environment variable values exist outside of memory, persisted to storage (disk, S3). ([289](https://github.com/nextstrain/cli/pull/289))

Bug fixes

* `nextstrain view` now waits (up to 10s) for Auspice to start responding before automatically opening it in the browser. This should eliminate the previous behaviour of sometimes opening the browser too soon. ([291](https://github.com/nextstrain/cli/pull/291))