Enhancements and Cleanup:
ISA MODEL
- The `model.py` file has been split into the `model` module. Each class has been split into a dedicated file.
- Added a method `from_dict()` to the ISA Investigation class to let users load objects from a dictionary.
- Added a method `to_dict()` to each ISA model class that let users dump ISA objects to dictionaries. (also see ISA JSONLD section)
- Added a new GraphQL interface to ISA investigation (see below querying isa for more information)
ISA TAB
- The `isatab.py` file has been split into the `isatab` module.
Functions and classes have been split into three submodules: `load` (to read isatab files), `dump` (to write isafiles) and `validate` (to run validation rules against an isatab file). Usage remains the same.
- The isatab validation pipeline have been reworked to allow the rules set to be extended with user custom rules. Please contact us if you would like to implement your own validation rules.
ISA JSON
- the `isajson.py` file has been split into the `isajson` module.
Functions and classes have been split into three files: `load.py`, `dump.py` and `validate.py`. Usage remains the same but there is now an alternative using dicts instead of files (see below)
- `load.py` and `dump.py` are now wrappers. The code of the isa json serializer/deserializer has been moved to be part of the ISA model. Users can still use the old `load()` and `dump()` methods but are now able to do it directly from an investigation object. Example
py
from json import load
from isatools.model import Investigation
isa_json_file = 'path/to/isa.json' the json file path
with open(isa_json_file, 'r') as f:
isa_json_input = load(f) load the dict from the file
investigation = Investigation() create en investigation
investigation.from_dict(isa_json_input) feed in the dictionary, will generated missing identifiers make inputs and outputs different
isa_json_output = investigation.to_dict() Dump it back to a dict
another_investigation = Investigation() create en investigation
another_investigation.from_dict(isa_json_output) feed in the dictionary
assert investigation == another_investigation assert equality between the created investigations.
assert investigation.to_dict() == another_investigation.to-dict() compare to_dict() methods
Note: identifiers may be missing from the json file and autogenerated by the ISA objects. In this case, output dictionnaries won't match which is why the investigation is being compared to another investigation.
ISA JSON-LD
- Using the new `to_dict()` function from model.py, it is possible to produce JSON-LD.
The The method takes an optional boolean parameter `ld` (default to `False`) that serializes to json-ld dictionaries by injecting `context`, `id` and `type` attributes. See isa json below for more information.
- Customisation of the JSON-LD serialization is achieved through the `set_context()` function of the `model` module.
- Added support for SIO, WD, OBO and SDO annotations (see https://github.com/ISA-tools/isa-api/tree/master/isatools/resources/json-context).
- Added support for generating URL identifiers (necessary for hosting triple stores).
- Check https://github.com/ISA-tools/isa-api/blob/master/isa-cookbook/content/notebooks/querying-isa-with-graphql-and-sparql.ipynb for examples.
SPARQL querying over ISA:
- The outputs of the ISA JSON LD serializer have been tested as a valid RDF representation. Using MTBLS instances with [rdflib](https://github.com/RDFLib/rdflib) and settting the contexts with `set_context(vocab='wd', all_in_one=False, local=False, include_contexts=False)` we were able to produce RDF triples and store them into a virtuosoDB and a berkeleyDB with SPARQL support.
- Check https://github.com/ISA-tools/isa-api/blob/master/isa-cookbook/content/notebooks/querying-isa-with-graphql-and-sparql.ipynb for examples.
GRAPHQL querying over ISA:
- Each ISA Investigation now has a graphQL interface bound to it.
A full documentation of the graphQL interface is available at https://github.com/ISA-tools/isa-api/tree/master/isatools/graphQL/README.md
MTBLS, the EMBL-EBI MetaboLights Client:
- Refactored the MTBLS client but kept the same functions (and signature) exposed. You can now use MTBLInvestigation instead of the old functions. Example:
py
from isatools.net.mtbls import MTBLSInvestigation
investigation = MTBLSInvestigation('MTBLS1')
This creates a temp dir for the investigation that gets deleted when the object is destroyed through the automatic garbage collector. You can pass a directory if you want to persist the investigation.
investigation = MTBLSInvestigation(mtbls_id='MTBLS1', output_directory="path/to/output/dir", output_format='tab')
You can then run all the old function through methods with the same name and chain methods without downloading the whole investigation again.
print(investigation.get_characteristics_summary())
print(investigation.get_factor_names())
ISA SQL:
- Created a relational SQL Alchemy model for ISA objects tested with PSQL and SQLite. All model files are available here: https://github.com/ISA-tools/isa-api/tree/master/isatools/database/models
Other
Versions update:
- Dropped python 3.6 and 3.7
- Added compatibility for python 3.10 (production ready)
- Added compatibility for python 3.11 (test only)
- Bump numpy, pandas and mzml.
Cookbook
- Added a cookbook for querying isa with sparql and grapql. See https://github.com/ISA-tools/isa-api/blob/master/isa-cookbook/content/notebooks/querying-isa-with-graphql-and-sparql.ipynb
Testing:
- The application has been further tested with CI, especially the model. Global coverage increased from 75% to 84%.
- Build time was reduced due to mocking some network operations.
ISA-API performance profiler:
- Added a small command line interface using cProfile to generate performance reports of different ISA features (in particular serialization and deserialization to tab and json).
Known issues:
- There is a know performance issue with the coveralls/coverage modules on python 3.11 making the build very slow (see https://github.com/nedbat/coveragepy/issues/1287). Performances for end users shouldn't be affected.
Contributors:
- Batista Dominique, University of Oxford (https://orcid.org/0000-0002-2109-489X)
- Philipoe Rocca Serra, University of Oxford (https://orcid.org/0000-0001-9853-5668)