Aiida

Improvements
- Add defaults for configure options of the `SshTransport` plugin [[4223]](https://github.com/aiidateam/aiida-core/pull/4223)
- `verdi status`: distinguish database schema version incompatible [[4319]](https://github.com/aiidateam/aiida-core/pull/4319)
- `SlurmScheduler`: implement `parse_output` to detect OOM and OOW [[3931]](https://github.com/aiidateam/aiida-core/pull/3931)

Features
- Make the RabbitMQ connection parameters configurable [[4341]](https://github.com/aiidateam/aiida-core/pull/4341)
- Add infrastructure to parse scheduler output for `CalcJobs` [[3906]](https://github.com/aiidateam/aiida-core/pull/3906)
- Add support for "peer" authentication with PostgreSQL [[4255]](https://github.com/aiidateam/aiida-core/pull/4255)
- Add the `--paused` flag to `verdi process list` [[4213]](https://github.com/aiidateam/aiida-core/pull/4213)
- Make the loglevel of the daemonizer configurable [[4276]](https://github.com/aiidateam/aiida-core/pull/4276)
- `Transport`: add option to not use a login shell for all commands [[4271]](https://github.com/aiidateam/aiida-core/pull/4271)
- Implement `skip_orm` option for SqlAlchemy `Group.remove_nodes` [[4214]](https://github.com/aiidateam/aiida-core/pull/4214)
- `Dict`: allow setting attributes through setitem and `AttributeManager` [[4351]](https://github.com/aiidateam/aiida-core/pull/4351)
- `CalcJob`: allow nested target paths for `local_copy_list` [[4373]](https://github.com/aiidateam/aiida-core/pull/4373)
- `verdi export migrate`: add `--in-place` flag to migrate archive in place [[4220]](https://github.com/aiidateam/aiida-core/pull/4220)

Bug fixes
- `verdi`: make `--prepend-text` and `--append-text` options properly interactive [[4318]](https://github.com/aiidateam/aiida-core/pull/4318)
- `verdi computer test`: fix failing result in harmless `stderr` responses [[4316]](https://github.com/aiidateam/aiida-core/pull/4316)
- `QueryBuilder`: Accept empty string for `entity_type` in `append` method [[4299]](https://github.com/aiidateam/aiida-core/pull/4299)
- `verdi status`: do not except when no profile is configured [[4253]](https://github.com/aiidateam/aiida-core/pull/4253)
- `ArithmeticAddParser`: attach output before checking for negative value [[4267]](https://github.com/aiidateam/aiida-core/pull/4267)
- `CalcJob`: fix bug in `retrieve_list` affecting entries without wildcards [[4275]](https://github.com/aiidateam/aiida-core/pull/4275)
- `TemplateReplacerCalculation`: make `files` namespace dynamic [[4348]](https://github.com/aiidateam/aiida-core/pull/4348)

Developers
- Rename folder `test.fixtures` to `test.static` [[4219]](https://github.com/aiidateam/aiida-core/pull/4219)
- Remove all files from the pre-commit exclude list [[4196]](https://github.com/aiidateam/aiida-core/pull/4196)
- ORM: move attributes/extras methods of frontend and backend nodes to mixins [[4376]](https://github.com/aiidateam/aiida-core/pull/4376)

Dependencies
- Dependencies: update minimum requirement `paramiko~=2.7` [[4222]](https://github.com/aiidateam/aiida-core/pull/4222)
- Depedencies: remove upper limit and allow `numpy~=1.17` [[4378]](https://github.com/aiidateam/aiida-core/pull/4378)

Deprecations
- Deprecate getter and setter methods of `Computer` properties [[4252]](https://github.com/aiidateam/aiida-core/pull/4252)
- Deprecate methods that refer to a computer's label as name [[4309]](https://github.com/aiidateam/aiida-core/pull/4309)

Changes
- `BaseRestartWorkChain`: do not run `process_handler` when `exit_codes=[]` [[4380]](https://github.com/aiidateam/aiida-core/pull/4380)
- `SlurmScheduler`: always raise for non-zero exit code [[4332]](https://github.com/aiidateam/aiida-core/pull/4332)
- Remove superfluous `ERROR_NO_RETRIEVED_FOLDER` from `CalcJob` subclasses [[3906]](https://github.com/aiidateam/aiida-core/pull/3906)

Bug fixes:
- Fix a file handle leak due to the `Runner` not closing the event loop if it created it itself [[4307]](https://github.com/aiidateam/aiida-core/pull/4307)
- `ArithmeticAddParser`: attach output before checking for negative value [[4267]](https://github.com/aiidateam/aiida-core/pull/4267)

Improvements
- Comprehensive restructuring and revamp of the online documentation [[4141]](https://github.com/aiidateam/aiida-core/pull/4141)
- Improve defaults for `verdi computer configure ssh` [[4055]](https://github.com/aiidateam/aiida-core/pull/4055)
- Provenance graphs: enable highlighting specific node classes (and highlight root node by default) [[4081]](https://github.com/aiidateam/aiida-core/pull/4081)

Performance
- Enable event-based monitoring of work chain child processes (they were being polled every second) [[4154]](https://github.com/aiidateam/aiida-core/pull/4154)
- Increase the default for `runner.poll.interval` config option from 1 to 60 seconds [[4150]](https://github.com/aiidateam/aiida-core/pull/4150)
- Increase the efficiency of the `SqlaGroup.nodes` iterator [[4094]](https://github.com/aiidateam/aiida-core/pull/4094)

Features
- Add a progress bar for export and import related functionality [[3599]](https://github.com/aiidateam/aiida-core/pull/3599)
- Enable loading config.yml files from URL in `verdi` commands with `--config` option [[3977]](https://github.com/aiidateam/aiida-core/pull/3977)
- `QueryBuilder`: add the `flat` argument to the `.all()` method [[3945]](https://github.com/aiidateam/aiida-core/pull/3945)
- `verdi status`: add `--no-rmq` flag to skip the RabbitMQ check [[4181]](https://github.com/aiidateam/aiida-core/pull/4181)
- Add support for process functions in `verdi plugin list` [[4117]](https://github.com/aiidateam/aiida-core/pull/4117)
- Allow profile selection in ipython magic `%aiida` [[4071]](https://github.com/aiidateam/aiida-core/pull/4071)
- Support more complex formula formats in `aiida.orm.data.cif.parse_formula` [[3954]](https://github.com/aiidateam/aiida-core/pull/3954)

Bug fixes
- `BaseRestartWorkChain`: do not assume `metadata` exists in inputs in `run_process` [[4210]](https://github.com/aiidateam/aiida-core/pull/4210)
- `BaseRestartWorkChain`: fix bug in `inspect_process` [[4166]](https://github.com/aiidateam/aiida-core/pull/4166)
- `BaseRestartWorkChain`: fix the "unhandled failure mechanism" for dealing with failures of subprocesses [[4155]](https://github.com/aiidateam/aiida-core/pull/4155)
- Fix exception handling in commands calling `list_repository_contents` [[3968]](https://github.com/aiidateam/aiida-core/pull/3968)
- Fix bug in `Code.get_full_text_info` [[4083]](https://github.com/aiidateam/aiida-core/pull/4083)
- Fix bug in `verdi daemon restart --reset` [[3969]](https://github.com/aiidateam/aiida-core/pull/3969)
- Fix tab-completion for `LinkManager` and `AttributeManager` [[3985]](https://github.com/aiidateam/aiida-core/pull/3985)
- `CalcJobResultManager`: fix bug that broke tab completion [[4187]](https://github.com/aiidateam/aiida-core/pull/4187)
- `SshTransport.gettree`: allow non-existing nested target directories [[4175]](https://github.com/aiidateam/aiida-core/pull/4175)
- `CalcJob`: move job resource validation to the `Scheduler` class fixing a problem for the SGE and LSF scheduler plugins [[4192]](https://github.com/aiidateam/aiida-core/pull/4192)
- `WorkChain`: guarantee to maintain order of appended awaitables [[4156]](https://github.com/aiidateam/aiida-core/pull/4156)
- Add support for binary files to the various `verdi` cat commands [[4077]](https://github.com/aiidateam/aiida-core/pull/4077)
- Ensure `verdi group show --limit` respects limit even in raw mode [[4092]](https://github.com/aiidateam/aiida-core/pull/4092)
- `QueryBuilder`: fix type string filter generation for `Group` subclasses [[4144]](https://github.com/aiidateam/aiida-core/pull/4144)
- Raise when calling `Node.objects.delete` for node with incoming links [[4168]](https://github.com/aiidateam/aiida-core/pull/4168)
- Properly handle multiple requests to threaded REST API [[3974]](https://github.com/aiidateam/aiida-core/pull/3974)
- `NodeTranslator`: do not assume `get_export_formats` exists [[4188]](https://github.com/aiidateam/aiida-core/pull/4188)
- Only color directories in `verdi node repo ls --color` [[4195]](https://github.com/aiidateam/aiida-core/pull/4195)

Developers
- Add arithmetic workflows and restructure calculation plugins [[4124]](https://github.com/aiidateam/aiida-core/pull/4124)
- Add minimal `mypy` run to the pre-commit hooks. [[4176]](https://github.com/aiidateam/aiida-core/pull/4176)
- Fix timeout in `tests.cmdline.commands.test_process:test_pause_play_kill` [[4052]](https://github.com/aiidateam/aiida-core/pull/4052)
- Revise update-dependency flow to resolve issue 3930 [[3957]](https://github.com/aiidateam/aiida-core/pull/3957)
- Add GitHub action for transifex upload [[3958]](https://github.com/aiidateam/aiida-core/pull/3958)

Deprecations
- The `get_valid_schedulers` class method of the `Scheduler` class has been deprecated in favor of `aiida.plugins.entry_point.get_entry_point_names` [[4192]](https://github.com/aiidateam/aiida-core/pull/4192)

In the fixing of three bugs, three minor features have been added along the way.

Features
- Add config option `daemon.worker_process_slots` to configure the maximum number of concurrent tasks each daemon worker can handle [[3949]](https://github.com/aiidateam/aiida-core/pull/3949)
- Add config option `daemon.default_workers` to set the default number of workers to be started by `verdi daemon start` [[3949]](https://github.com/aiidateam/aiida-core/pull/3949)
- `CalcJob`: make submit script filename configurable through the `metadata.options` [[3948]](https://github.com/aiidateam/aiida-core/pull/3948)

Bug fixes
- `CalcJob`: fix bug in idempotency check of upload transport task [[3948]](https://github.com/aiidateam/aiida-core/pull/3948)
- REST API: reintroduce CORS headers, the lack of which was breaking the Materials Cloud provenance explorer [[3951]](https://github.com/aiidateam/aiida-core/pull/3951)
- Remove the equality operator of `ExitCode` which caused the serialization of workchains to fail if put in the workchain context [[3940]](https://github.com/aiidateam/aiida-core/pull/3940)

Deprecations
- The `hookup` argument of `aiida.restapi.run_api` and the `--hookup` option of `verdi restapi` are deprecated [[3951]](https://github.com/aiidateam/aiida-core/pull/3951)

Features
- `ExitCode`: make the exit message parameterizable through templates [[3824]](https://github.com/aiidateam/aiida-core/pull/3824)
- `GroupPath`: a utility to work with virtual `Group` hierarchies [[3613]](https://github.com/aiidateam/aiida-core/pull/3613)
- Make `Group` sub classable through entry points [[3882]](https://github.com/aiidateam/aiida-core/pull/3882)[[#3903]](https://github.com/aiidateam/aiida-core/pull/3903)[[#3926]](https://github.com/aiidateam/aiida-core/pull/3926)
- Add auto-complete support for `CodeParamType` and `GroupParamType` [[3926]](https://github.com/aiidateam/aiida-core/pull/3926)
- Add export archive migration for `Group` type strings [[3912]](https://github.com/aiidateam/aiida-core/pull/3912)
- Add the `-v/--version` option to `verdi export migrate` [[3910]](https://github.com/aiidateam/aiida-core/pull/3910)
- Add the `-l/--limit` option to `verdi group show` [[3857]](https://github.com/aiidateam/aiida-core/pull/3857)
- Add the `--order-by/--order-direction` options to `verdi group list` [[3858]](https://github.com/aiidateam/aiida-core/pull/3858)
- Add `prepend_text` and `append_text` to `aiida_local_code_factory` pytest fixture [[3831]](https://github.com/aiidateam/aiida-core/pull/3831)
- REST API: make it easier to call `run_api` in wsgi scripts [[3875]](https://github.com/aiidateam/aiida-core/pull/3875)
- Plot bands with only one kpoint [[3798]](https://github.com/aiidateam/aiida-core/pull/3798)

Bug fixes
- Improved validation for CLI parameters [[3894]](https://github.com/aiidateam/aiida-core/pull/3894)
- Ensure unicity when creating instances of `Autogroup` [[3650]](https://github.com/aiidateam/aiida-core/pull/3650)
- Prevent nodes without registered entry points from being stored [[3886]](https://github.com/aiidateam/aiida-core/pull/3886)
- Fix the `RotatingFileHandler` configuration of the daemon logger[[3891]](https://github.com/aiidateam/aiida-core/pull/3891)
- Ensure log messages are not duplicated in daemon log file [[3890]](https://github.com/aiidateam/aiida-core/pull/3890)
- Convert argument to `str` in `aiida.common.escaping.escape_for_bash` [[3873]](https://github.com/aiidateam/aiida-core/pull/3873)
- Remove the return statement of `RemoteData.getfile()` [[3742]](https://github.com/aiidateam/aiida-core/pull/3742)
- Support for `BandsData` nodes without `StructureData` ancestors [[3817]](https://github.com/aiidateam/aiida-core/pull/3817)

Deprecations
- Deprecate `--group-type` option in favor of `--type-string` for `verdi group list` [[3926]](https://github.com/aiidateam/aiida-core/pull/3926)

Documentation
- Docs: link to documentation of other libraries via `intersphinx` mapping [[3876]](https://github.com/aiidateam/aiida-core/pull/3876)
- Docs: remove extra `advanced_plotting` from install instructions [[3860]](https://github.com/aiidateam/aiida-core/pull/3860)
- Docs: consistent use of "plugin" vs "plugin package" terminology [[3799]](https://github.com/aiidateam/aiida-core/pull/3799)

Developers
- Deduplicate code for tests of archive migration code [[3924]](https://github.com/aiidateam/aiida-core/pull/3924)
- CI: use GitHub Actions services for PostgreSQL and RabbitMQ [[3901]](https://github.com/aiidateam/aiida-core/pull/3901)
- Move `aiida.manage.external.pgsu` to external package `pgsu` [[3892]](https://github.com/aiidateam/aiida-core/pull/3892)
- Cleanup the top-level directory of the repository [[3738]](https://github.com/aiidateam/aiida-core/pull/3738)
- Remove unused `orm.implementation.utils` module [[3877]](https://github.com/aiidateam/aiida-core/pull/3877)
- Revise dependency management workflow [[3771]](https://github.com/aiidateam/aiida-core/pull/3771)
- Re-add support for Coverage reports through codecov.io [[3618]](https://github.com/aiidateam/aiida-core/pull/3618)

Changes
- Emit a warning when input port specifies a node instance as default [[3466]](https://github.com/aiidateam/aiida-core/pull/3466)
- `BaseRestartWorkChain`: require process handlers to be instance methods [[3782]](https://github.com/aiidateam/aiida-core/pull/3782)
- `BaseRestartWorkChain`: add method to enable/disable process handlers [[3786]](https://github.com/aiidateam/aiida-core/pull/3786)
- Docker container: remove conda activation from configure-aiida.sh script [[3791]](https://github.com/aiidateam/aiida-core/pull/3791)
- Add fixtures to clear the database before or after tests [[3783]](https://github.com/aiidateam/aiida-core/pull/3783)
- `verdi status`: add the configuration directory path to the output [[3587]](https://github.com/aiidateam/aiida-core/pull/3587)
- `QueryBuilder`: add support for `datetime.date` objects in filters [[3796]](https://github.com/aiidateam/aiida-core/pull/3796)

Bug fixes
- Fix bugs in `Node._store_from_cache` and `Node.repository.erase` that could result in calculations not being reused [[3777]](https://github.com/aiidateam/aiida-core/pull/3777)
- Caching: fix configuration spec and validation [[3785]](https://github.com/aiidateam/aiida-core/pull/3785)
- Write migrated config to disk in `Config.from_file` [[3797]](https://github.com/aiidateam/aiida-core/pull/3797)
- Validate label string at code setup stage [[3793]](https://github.com/aiidateam/aiida-core/pull/3793)
- Reuse `prepend_text` and `append_text` in `verdi computer/code duplicate` [[3788]](https://github.com/aiidateam/aiida-core/pull/3788)
- Fix broken imports of `urllib` in various locations including `verdi import` [[3767]](https://github.com/aiidateam/aiida-core/pull/3767)
- Match headers with actual output for `verdi data structure list` [[3756]](https://github.com/aiidateam/aiida-core/pull/3756)
- Disable caching for the `Data` node subclass (this should not affect usual caching behavior) [[3807]](https://github.com/aiidateam/aiida-core/pull/3807)