Miniwdl

`miniwdl configure`

This new subcommand walks through generating a runner configuration file with common opt-in features, like the call and download caches. It eases first-time setup, allowing further customization.

`miniwdl run`

* Improved macOS compatibility, though setup still has a few roadbumps (https://github.com/chanzuckerberg/miniwdl/issues/145)
* The runner "flocks" the top-level run directory, in addition to the log file therein, while the run is in progress.
* Recognize `runtime.container` as an alias for `runtime.docker`
* Adds plugin extensibility for call cache (yet to be documented)

`miniwdl check`

* Startup latency significantly improved (~50%)

`WDL`

* WDL `version 1.1` is now recognized, anticipating [its ratification in the OpenWDL community](https://github.com/openwdl/wdl/pull/428)
* Roughly, this includes all of miniwdl's existing `version development` features *except* `Directory` support
* Python scripts now must explicitly import the `WDL.runtime` and `WDL.Lint` subpackages in order to use their functionality, in addition to `import WDL` (for CLI startup latency improvement)
* `Float` coerces to `String` with six decimal places, per spec

* Adds `read_object()` and `read_objects()`
* *only* for initializing `Map[String,String]` and structs
* `Object` remains generally unsupported
* Initialize structs from `Map[String,_]` returned by `read_json()` or `read_object()`
* liable to fail at runtime if types don't match
* new lint warning `UnverifiedStruct`
* String literals may contain both escape sequences and embedded, non-ASCII UTF-8 characters
* Call cache: store & reuse prior [sub]workflow outputs, as well as previously-supported task outputs
* Resolve docker image tag (e.g. `ubuntu:20.04`) to RepoDigest (`ubuntusha256:c95a8e48...`) before scheduling
* Ensures use of identical image across retries or swarm workers
* (`version development`) [Abbreviated syntax for passthrough call inputs](https://github.com/openwdl/wdl/pull/365)
* `call t { input: x, y=z }` is shorthand for `call t { input: x=x, y=z }`

"WDL 2.0" preview

This release adds new features defined in the pending ["WDL 2.0" language version](https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md). For WDL documents beginning with a `version development` declaration, the parser and runner support:

* `Directory` inputs and outputs for tasks and workflows
* `Directory` downloads and caching for `s3:` URIs (other services pending)
* `None` literal keyword, to create an empty value explicitly
* `call task1 after prevCall { ... }` to declare a dependency between workflow calls without an output-input linkage
* [Revised struct literals](https://github.com/openwdl/wdl/pull/297) declaring the struct type name instead of `object`, e.g. `Car { make: "Ford", model: "Model T" }`
* New standard library functions:
* `min()` `max()`
* `sep()` `quote()` `squote()`
* `keys()` `as_map()` `as_pairs()` `collect_by_key()`

⚠️ `version development` is a moving target, as "WDL 2.0" is still to be finalized in the OpenWDL community.

`miniwdl run`

* CLI:
* `key=value` inputs accept a space after the `=` and before the value, allowing shell path autocompletion for File/Directory inputs
* `--empty` can specify empty strings (as well as empty arrays)
* `--none` to nullify optional inputs with defaults
* The legacy `output_links/` (linking to `out/`) is no longer created in each run directory
* Allow task call caching with URI inputs
* Default docker image (used if `runtime.docker` isn't specified) updated to `ubuntu:20.04`
* Call and download cache entries created with previous releases are invalidated, and may be deleted

`WDL`

* `Pair` values serialize to & from JSON `{"left": x, "right": y}` instead of `[x,y]`
* Allow `Map` literals with compound key types

`miniwdl run`

* Task call caching (experimental): cache entries are automatically invalidated if any referenced input/output files were removed or modified

`miniwdl run`

File URI download improvements:

* Allow tasks' as well as workflows' File inputs to declare default URI e.g. `File an_input = "https://..."`
* Use this feature with caution, as it's not explicitly allowed (nor disallowed) by the WDL specification; setting URIs in JSON inputs is more portable
* Prevent run failures caused by multiple concurrent attempts to download the same URI
* Enable download of public S3 objects even if operative IAM policy prevents access to the containing bucket
* Workflow calls are cached based on original input URIs instead of temporary/cached local copy filename

`miniwdl check`

* Add warning for unbound declarations outside of task/workflow `input{}` sections in WDL 1.0 documents

`miniwdl run`

* Polish CLI with `parameter_meta` and defaults for input help, ANSI colors, and clarified error messages (dpark01 slzarate)
* Fix bugs handling input/output files with unusual characters in their names (409 410 415)
* `--error-json` reports errors in parsing WDL source code or validating inputs as well as execution (384 MarkAZhang)
* Call caching: refined task digest algorithm to ignore WDL comments and most whitespace outside of command script
* Effectively invalidates existing cache entries (no action required; fine to delete them)
* More such invalidations are likely to come, as the feature continues maturing
* Legacy note: The `out/` folder of output file links found in each run folder was called `output_links/` prior to v0.8.0. Currently a symlink `output_links/ -> out/` is generated as well, but this will be discontinued in the future.

`WDL`

* Improved type unification algorithm for compound types (406)

meta

* Relax requirements.txt version constraints (413 TMiguelT)
* Add flake8 linter (407 kislyuk)