Datalad-crawler

Latest version: v1.0.2

Safety actively analyzes 630523 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 5 of 5

0.4.1

- Compatibility layer with 0.12 series of DataLad changing API
(no backend option for `create`)

0.4

Primarily a variety of fixes and small enhancements. The only notable
change is stripping away testing/support of git-annex direct mode.

- do not depend on a release candidate of the DataLad, since PIP then opens the
way to a RCs for any later releases to be installed
- `simple_with_archives`
- issue warning if incoming_pipeline has Annexificator but no `annex` is given
- `crcns`
- skip (but warn if relevant) records without xml
- do not crash while saving updated crawler's URL db to the file which is annexed.

0.3

Primarily a variety of fixes

- `crcns` crawler now uses new datacite interface
- `openfmri` crawler uses legacy.openfmri.org
- `simple_with_archives`
- by default now also match pure .gz files to be downloaded
- `archives_re` option provides regex for archives files (so `.gz`
could be added if needed)
- will now run with `tarballs=False`
- `add_annex_to_incoming_pipeline` to state either to add `annex`
to the incoming pipeline
- new `stanford_lib` pipeline
- aggregation of metadata explicitly invokes incremental mode
- tests
- variety of tests lost their `known_failure_v6` and now tolerant
to upcoming datalad 0.11.2

0.2

- All non-master branches in the pipelines now will initiate from master
branch, not detached. That should allow to inherit .gitattributes
settings of the entire dataset

0.1

- First release as a DataLad extension. Functionality remains identical
to DataLad 0.10.0.rc2

Page 5 of 5

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.