Nextstrain-augur

Features

* filter: Added a new option `--max-length` to filter out sequences that are longer than a certain amount of base pairs. [1429][] (victorlin)
* parse: Added support for environments that use pandas 2.x. [1436][] (emollier, victorlin)

Bug Fixes

* filter: Updated docs with an example of tiered subsampling. [1425][] (victorlin)
* export: Fixes bug [1433] introduced in v23.1.0, that causes validation to fail when gene names start with `nuc`, e.g. `nucleocapsid`. [1434][] (corneliusroemer)
* import: Fixes bug introduced in v24.2.0 that prevented `import beast` from running. [1439][] (tomkinsc)
* translate, ancestral: Compound CDS are now exported as segmented CDS and are now viewable in Auspice. [1438][] (jameshadfield)

[1425]: https://github.com/nextstrain/augur/pull/1425
[1429]: https://github.com/nextstrain/augur/pull/1429
[1433]: https://github.com/nextstrain/augur/issues/1433
[1434]: https://github.com/nextstrain/augur/pull/1434
[1436]: https://github.com/nextstrain/augur/pull/1436
[1438]: https://github.com/nextstrain/augur/pull/1438
[1439]: https://github.com/nextstrain/augur/pull/1439

Bug Fixes

* filter: Updated the help and report text of `--min-length` to explicitly state that the minimum length filter only counts standard nucleotide characters A, C, G, or T (case-insensitive). This has been the behavior since version 3.0.3.dev1, but has never been explicitly documented. [1422][] (joverlee521)
* frequencies: Fixed a bug introduced in 24.2.0 and 24.1.0 that prevented `--regions` from working when providing regions other than the default "global" region. [1424]

[1422]: https://github.com/nextstrain/augur/pull/1422
[1424]: https://github.com/nextstrain/augur/pull/1424

Bug Fixes

* filter: In versions 24.2.0 and 24.2.1, `--query` stopped working in cases where internal optimizations added in version 24.2.0 failed to parse the columns from the query. It now falls back to non-optimized behavior that allows queries to work. [1418][] (victorlin)
* filter: Handle backtick quoting in internal optimizations of `--query`. [1417][] (victorlin)

[1417]: https://github.com/nextstrain/augur/pull/1417
[1418]: https://github.com/nextstrain/augur/pull/1418

Bug Fixes

* frequencies: Fixed a bug introduced in 24.2.0 that prevented `--method diffusion` from working alongside `--tree`. [1412][] (victorlin)

[1412]: https://github.com/nextstrain/augur/issues/1412

Features

* filter: Added a new option `--query-columns` that allows specifying what columns are used in `--query` along with the expected data types. If unspecified, automatic detection of columns and types is attempted. [1294][] (victorlin)
* `augur.io.read_metadata`: A new optional `columns` argument allows specifying a subset of columns to load. The default behavior still loads all columns, so this is not a breaking change. [1294][] (victorlin)
* `augur parse`: A new optional `--output-id-field` argument allows the user to select any ID field for the produced FASTA file (e.g. 'accession' instead of 'name' or 'strain'). [1403][] (j23414)
* When no `--output-id-field` is given and the data has both `name` and `strain` fields, continue to preferentially use `name` over `strain` as the sequence ID field; but, throw a deprecation warning that the order will be switched to prefer `strain` over `name` in the future to be consistent with the rest of Augur.
* Added entry to [DEPRECATED.md](./DEPRECATED.md).
* Compression should now be supported for all input and output files. Please [open an issue](https://github.com/nextstrain/augur/issues) if you find one that doesn't! [#1381][] (victorlin)
* export v2: Add support to specify metadata columns to export without using them as colorings. This can be done with the `metadata_columns` property in the Auspice config JSON or via the `--metadata-columns` flag in the command line. [1384][] (joverlee521)

Bug Fixes

* filter: In version 24.1.0, automatic conversion of boolean columns was accidentally removed. It has been restored with additional support for empty values evaluated as `None`. [1410][] (victorlin)
* filter: The order of rows in `--output-metadata` and `--output-strains` now reflects the order in the original `--metadata`. [1294][] (victorlin)
* filter, frequencies, refine: Performance improvements to reading the input metadata file. [1294][] (victorlin)
* For filter, this comes with increased writing times for `--output-metadata` and `--output-strains`. However, net I/O speed still decreased during testing of this change.
* filter: Updated the help text of `--include` and `--include-where` to explicitly state that this can add strains that are missing an entry from `--sequences`. [1389][] (victorlin)
* filter: Fixed the summary messages to properly reflect force-inclusion of strains that are missing an entry from `--sequences`. [1389][] (victorlin)
* filter: Updated wording of summary messages. [1389][] (victorlin)
* Enforce UTF-8 encoding when reading and writing files. Improve error messages when a non-UTF-8 file is used. [1381][] (victorlin)

[1294]: https://github.com/nextstrain/augur/pull/1294
[1381]: https://github.com/nextstrain/augur/pull/1381
[1384]: https://github.com/nextstrain/augur/pull/1384
[1389]: https://github.com/nextstrain/augur/pull/1389
[1410]: https://github.com/nextstrain/augur/pull/1410
[1403]: https://github.com/nextstrain/augur/pull/1403

Features

* `augur.io.read_metadata`: A new optional `dtype` argument allows custom data types for all columns. Automatic type inference still happens by default, so this is not a breaking change. [1252][] (victorlin)
* `augur.io.read_vcf` has been removed and usage replaced with TreeTime's function of the same name which has improved validation of the VCF file. [1366][] (jameshadfield)

Bug Fixes

* filter, frequencies, refine: Speed up reading of the metadata file. [1252][] (victorlin)
* traits: Previously, columns with only numeric values were treated as numerical data. These are now treated as categorical data for discrete trait analysis. [1252][] (victorlin)
* Support Biopython `≥1.82` by requiring bcbio-gff `≥0.7.1`. [1400][] (victorlin)

[1252]: https://github.com/nextstrain/augur/pull/1252
[1366]: https://github.com/nextstrain/augur/pull/1366
[1400]: https://github.com/nextstrain/augur/pull/1400