Law

New features

- Add load and dump support for directory targets. (fa18d328)
- Add localization support for directory targets. (21082952)
- Add TFSavedModelFormatter to tensorflow contrib. (78e5ef71)
- Add DaskAwkwardFormatter to awkward contrib. (a36219ac)
- Add notebook example. (e92fb58d)

Improvements

- Forward some requests from remote to local fs. (585e3de5)
- Refactor parquet file merging in pyarrow contrib. (f13d6a5e)
- Improve %ilaw ipython magic. (e4a213aa)
- Parse left-over string parameters. (b1abe377)

Breaking changes

*None*.

Fixes

- Fix legacy getargspec usage. (097245cc)
- Fix stat messages after file merigng in root contrib. (d9bc203d)
- Fix logging on colab. (b8d00e3a)
- Reset parser objects after law_run util. (8105deec)

Features

- Add parameter copy helper. (118f977)
- Make c8-py310 image default one. (eef8289, a84fea3)

Fixes & improvements

- Fix interactive output of tasks' fetch_output. (02dc374)
- Improve branch boundary updates in workflows. (5ba9d74)
- Disable instance caching for proxy tasks. (❗️a2a8e49) (**Note:** this fix solves issues with sandboxes complaining about `LAW_SANDBOX_WORKER_ID` not being set)

Features

- Add OptionalBoolParameter for transparently handling None's. (9c2e0d6)
- Update examples. (b9adec9, d7d8e9a)
- Allow and invoke auto load hooks in contrib packages. (2f6246e)
- Pass-through multi-core setting to remote jobs. (🎉, 70a223f)
- Allow non-parameter values in task repr. (f98d92d)

Fixes & improvements

- Fix remote target cache invalidation. (❗️, 239454d)
- Enforce abspaths in contrib packages. (dbcc1b5)
- Fix prog in %ilaw ipython magic, use auto_load. (fa84765)
- Fix interactive task output. (c845aeb)

Features

- Add smart line breaks to interactive task outputs by default. (cdd2f67a)
- Refine interactive task messages. (56fb7fdd)
- Add sync mechanism between env and law config. (faa032ce)
- Add davix error parsers to gfal remote interface. (dc8a78e6)
- Recursively flatten target collections in fetch_output handler. (4230365a)
- Add centos8 images based on conda. (a8d9a053, 06212e64, 389bbb72)
- Use lzma-devel in docker images. (1e2b4b08)
- Add sequential_htcondor_at_cern example. (ddab2d5d)
- Refactor base path handling in local file system. (e3fb0252)
- Add wrap method to DotDict. (e8a37f81)
- Allow configuring base paths for local file systems. (438212ae)
- Add target formatter loading python scripts. (baa112d0)
- Add names to file systems. (b38dab27)
- Add gzip target formatter. (b6db2926)


Fixes & improvements

- Fix typo in 'localize' decorator. (42723ded)
- Better interface with dynamic dependencies in forest merge contrib task. (c45f5e93)
- Fix config file inheritence and extensions. (7756cb61)
- Minor cleanup of remote workflow examples. (ab27ab44)
- Fix DotDict for legacy python 2 support. (22898ab3)
- Refine zsh compatibility. (a87212ec, ff3aa35b)
- Propagate fs between local targets in parent and child methods. (fbba65c7)
- Improve auto completion on zsh. (e2f04b2e)
- Fix querying slurm jobs after they're gone (470073ef)

Features and improvements

Breaking

- Fully remove --start-branch and --end-branch from workflows. (a03e7c8f)

Major

- Speedup SiblingFileCollection removal. (be1317ca)
- Add mtime patience feature for remote caches. (4a2dac9a)
- Allow csv-style double-quote quoting in (Multi)CSVParameter. Close 123. (eb8536b2)
- Refactor target inheritance structure. (44e0f0ea)
- Disentangle copying and rendering of job input files. (fe378a6a)
- Add ParquetTableFormatter to pyarrow contrib. (57ad33cc)

Minor

- Unified bash variable usage. (22cf9fe2)
- Support output placeholders in ForestMerge contrib task. (9dc0c538)
- Hook logger setup into get_logger(). (7c9b604b)
- Yield DynamicRequirements in local wf if available. (0ac39c78)
- Show exit code in failed sandbox env commands. (0c62aa29)
- Allow returning keys in target collections. (99e47dc5)
- Generalize forest merge inputs. (72d9f184)
- Allow sanbox and workflow objects to be setup lazily. (f8a4fefe)
- Favor perf_counter() over time() where appropriate. (409f431a)
- Allow skipping run method decorators. (b460f42f)
- Consistent handling of executable in all job interfaces. (fee7202c)
- Allow setting root task manually. (74636539, 83760342)
- Add inplace option to util.merge_dicts. (1cda4432)
- Add JobInputFile. (1a3a6def)
- Use JobInputFile in arc, glite, slurm and lsf workflows. (e13d3698)
- Add rendering error. (35ab3e0f)
- Update docker images 39 and 310. (81c9afb4)
- Allow control over merge forest caching in ForestMerge. (8188f8f6)
- Refine workflow completion hook. (21cdb593)
- Filter unknown parameters after modify_param_values. (5798f342)
- Refine proxy setup in sandbox and workflow tasks. (7f0cb242)
- Safe map -> list conversion. (936c1d11)
- Properly log sandbox commands. (01ef2b93)
- Add split_csv option to all config getters. (e5c52cf8)
- Make the coffea contrib formatter understand parquet and root. (ad32e33a)

Fixes

- Support htcondor cluster submission. (313f31db)
- Support law3 executable. (f42a07b2, 344361fd)
- Fix namespaces in py3. (5018a76e)
- Fix sandbox command concat. (69fb5b04)
- Make slurm job file executable. (e56a7616)
- Take submodules into account for calculating repo checksums. (2e646b7b)
- Fix improved target collection removal. (c4ec1b8d)
- Refine command joining in sandboxes. (9b13f911)
- Fix base names of non-copied input files in remote job interfaces. (3b842935)
- Drop run_decorators feature from ForestMerge. (5c5ddc9d)
- Proper float format. (90ab5760)
- Fix workflow proxy instantiation. (14cd58ef)
- Fix slurm executable. (006a6c61)
- Fix executable paths, adjust slurm peculiarity (118). (3fed8914)
- Fix ambiguity between submission-relatve paths and actual basenames. (8b848d7c)
- Fix branch repr. (1ac00323)
- Fix input files in render variables. (f24e837b)
- Fix LocalTarget.__del__ for legacy 2.7 support. (a1457b3c)
- Fix GFAL2 URL (7f1bcd92)
- Favor task id in TaskParameter serialization. (6a650ad4)
- Add modify_param_args and modify_param_values to BaseTask for correct super mro's. (69a2c2b5)
- Fix range_expand and range_join utils. (ecdc0a3f)
- Fix branch setting in contrib ForestMerge task. (732da23c)
- Fix cache cleanup. (11f68242)
- Fix remote target cache cache cleanup. (b10c4fbd)
- Fix parquet writer version in pyarrow contrib. (d40b5333)

Features and Improvements

- Add VenvSandbox. (5321b50)
- Slurm workflows 🎉 (5b05715)
- Refactor job file handling in existing remote workflows (39cc5a7, 6dab50c, a400ff6, 9251c3a, 38bae13, 81ddfa6)
- Add awkward contrib package. (9f44202)
- Add dump() to uproot formatter. (8da8e6d)
- Add util.InsertableDict. (f88eeb6)
- Add iter_progress to base task. (e17fa05)
- Refactor human time parsing. (8625955)
- Add BytesParameter. (402201f)
- Optionally add file size to `FileSystemTarget` (114, 5e6153d, 7399a3b)
- Update docker files to latest deps (22faf15, a06d888)
- Add default set of prefered cli parameters to BaseTask. (c8f4589)


Fixes

- Fix missing fetch_output support for target collections. (010f379)
- Fix env encoding in bash sandbox. (87203fd)
- Fix log flushing in sandboxes. (b20629f)