Indra

Input processors
- New PhosphoELM API and processor in `indra.sources.phosphoelm`.
- PyBEL API and processor updated for PyBEL 0.14 compatibility in `indra.sources.bel`.
- Improvements in regrounding approach with Eidos in `indra.sources.eidos`.
- Bug fixes to offline reading via the REACH API in `indra.sources.reach`.
- Refactor ISI reader API and processor for containerization in `indra.sources.isi`.

Core assembly modules
- Full integration of Gilda disambiguation with grounding mapper both as
a package and a web service in `indra.preassembler.grounding_mapper`.
- Refactor grounding mapper module to separate out standardization functions
in `indra.preassembler.grounding_mapper.standardize`.

Output assemblers
- Many improvements to the HTML assembler including refactoring,
improved grouping and heading generation, and support Influences in
`indra.assemblers.html`.
- Improvements to grounding and introduction of part_of relations in
PyBEL assembler in `indra.assemblers.pybel`.


Resources
- Integrate a set of known MeSH to GO mappings.

Explanation
- Improvements to reporting paths in the explanation module in
`indra.explanation`.

Tools
- Many new features in the Live curation, including actively loading
and saving corpora on S3 in `indra.tools.live_curation`.
- Add new high-throughput reader management for ISI and TRIPS readers
in `indra.tools.reading`
- Significant refactoring of `indra.tools.reading` to separate reader
classes and improve containerization.

Package structure
- Remove the top-level data folder from version control (not part of package)
- Make INDRA compatible with networkx 2.4 (which removed several deprecated
features).

Input processors
- Refactoring of the BEL input API in `indra.sources.bel`.
Access to Selventa corpora is now via https://github.com/cthoyt/selventa-knowledge.
- Multitude of improvements to the CWMS processor in `indra.sources.cwms`,
including extracting migration events.
- Refactoring of INDRA DB REST API in `indra.sources.indra_db_rest` to use
a class-based processor structure for all endpoints.
- Add Gilda grounding option to the ISI reader in `indra.sources.isi`.

Core assembly modules
- Normalization across equivalent and opposite concepts in `indra.preassembler`
based on an ontology.
- Improved custom preassembly functions for location and time matching in
`indra.preassembler.custom_preassembly`.
- Preliminary integration of a Gilda client with Grounding mapper in
`indra.preassembler.grounding_mapper.gilda`.

Output assemblers
- Improved headings and grounding display in the HTML assembler in
`indra.assemblers.html`.
- Generalize INDRA Net assembler to causal influences and associations in
`indra.assemblers.indranet`.
- Make PySB assembler compatible with PySB 1.9 in `indra.assemblers.pysb`.

Resources
- Integrate new WorldModelers ontologies with INDRA in
`indra.preassembler.make_wm_ontologies`.
- Updates to FamPlex and other grounding resources.

Representation
- New MovementContext class to represent movement time/places for Events
- New QuantitativeState class to represent specific quantities associated with Events

Input sources
- Support for extracting Migration statements from CWMS in `indra.sources.cwms`
- Adaptation to changes in Eidos API and output format in `indra.sources.eidos`
- Adaptation to changes in Hume API and output format in `indra.sources.hume`
- Custom processor for migration tables in `indra.sources.eidos.migration_table_processor`
- Support for representing source counts in INDRA DB REST in `indra.sources.indra_db_res`

Internal assembly
- Significantly improved and generalized DB refs name standardization, and preassembly matching/refinement for all entities including chemicals and biological processes
- New custom matches/refinement function feature in Preassembler in `indra.preassembler` and BeliefEngine in `indra.belief`
- Support for new World Modelers ontology
- Grounding mapper refactored with new misgrounding map functionality in `indra.preassembler.grounding_mapper`

Model assemblers
- New IndraNet assembler to assemble custom networks for path finding in `indra.assemblers.indranet`
- New NiceCxAssembler for streamlined CX network assembly in `indra.assemblers.cx`
- Significantly improved HTML assembler with source count badges and support for templates `indra.assemblers.html`
- Refactored index card assembler with extended metadata support in `indra.assemblers.index_card`

Database/literature clients
- New PubChem client in `indra.databases.pubchem_client`
- New miRBase client in `indra.databases.mirbase_client`
- New Experimental Factor Ontology (EFO) client in `indra.databases.efo_client`
- New Phenotype Ontology (HP) client in `indra.databases.hp_client`
- New Disease Ontology (DOID) client in `indra.databases.doid_client`
- New mapping functionalities in HGNC, MeSH and ChEBI clients
- ScienceDirect search feature added in Elsevier client in `indra.literature.elsevier_client`
- Additional article info extraction and support for getting MeSH annotations in PubMed client in `indra.literature.pubmed_client`.


Model analysis
- Refactored ModelChecker with four new subclasses for analysis of specific model/network types in `indra.explanation`

Tools
- S3 corpus upload feature in LiveCuration tool in `indra.tools.live_curation`

Representation
- INDRA Events introduced as arguments of Influences or as standalone statements
- Influence statements are no longer derived from IncreaseAmount
- New object model for representing Delta/QualitativeDelta in Events

Input processors
- Support for standalone Events and Migration events extracted from CWMS in `indra.sources.cwms`
- Support for extracting standalone Events from Eidos in `indra.sources.eidos`
- Support for extracting standalone Events from Hume in `indra.sources.hume`
- Support for extracting standalone Events from Sofia in `indra.sources.sofia`

Database/Literature clients
- Support for determining type of gene in `indra.databases.hgnc_client`
- Support for mapping outdated gene names to current ones in `indra.databases.hgnc_client`
- Improved extraction of text from Elsevier XMLs in `indra.literature.elsevier_client`
- Additional annotations and metadata extraction support in `indra.literature.pubmed_client`

Dependencies
- Use Open JDK instead of Oracle JDK with pyjnius
- `deft` renamed to `adeft`

Assemblers
- All relevant assemblers adapted to new Influence/Event/Concept representation
- Support for generating Kappa influence and contact maps in PySB assembler in `indra.assemblers.pysb`
- Support for Influence statements in CX assembler in `indra.assemblers.cx`

Model analysis
- Additional logic in ModelChecker for Kappa influence map pruning in `indra.explanation.model_checker`

Reading tools
- Refactoring of AWS-based reading tools

Input processors
- New input API and processor for HPRD in `indra.sources.hprd` to access
http://www.hprd.org/
- New input API and processor for TRRUST in `indra.sources.trrust` to access
https://www.grnpedia.org/trrust/
- Major overhaul and improvement of MedScan API and processor in
`indra.sources.medscan`
- Extensions and improvements in RLIMS-P processor including capability to
process dumped result files in `indra.sources.rlimsp`
- Fixes to adapt to latest SIGNOR content in `indra.sources.signor`
- New API endpoint to process Causal Biological Networks from http://causalbionet.com in `indra.sources.bel`

Database clients
- New resources and functions to map ChEBI names to and from IDs in `indra.databases.chebi_client`
- New resources and functions to map MeSH names to and from IDs in `indra.databases.mesh_client`

Model analysis
- Bug fixes and improvements in `indra.explanation.model_checker`

General
- Optimized pickle protocols used to exploit Python 3+ compatibility

Python compatibility
- This release of INDRA is compatible with Python 3.5+ only and 2.x is not supported anymore.

Package structure, dependencies
- The `protmapper` package is a new dependency which takes over interfacing with UniProt (including file-based access and web service access). Consequently, almost all functionalities of `indra.databases.uniprot_client` and `indra.databases.phosphosite_client`, as well as the core of `indra.preassembler.site_mapper` are now provided by `protmapper`. The overall API of these INDRA modules has not changed, however.
- INDRA is now also available as a Docker image on Dockerhub under
`labsyspharm/indra`, further instructions in the README
- Installation instructions for the Python-Java bridge pyjnius have been updated,
INDRA's Java/Scala based dependencies can now work with versions of Java up to 11.
- The `statements.py` file has been refactored into its own module in
`indra.statements`, the overall API has not changed.
- INDRA's logging is now fully hierarchical and follows the submodule
structure.

Input processors
- CWMS processor in `indra.sources.cwms` can extract temporal and geo-spatial
context.
- Major overhaul of Hume processor in `indra.sources.hume`, supporting latest
JSON-LD format and extracting temporal and geo-spatial context
- An additional JSON-based API and processor is now available for SOFIA in
`indra.sources.sofia`, supporting text reading.
- New input processor for the RLIMS-P reading system in `indra.sources.rlimsp`.
- The INDRA DB REST source in `indra.sources.indra_db_rest` has been refactored
into multiple files and now provides a Processor class-based interface similar
to other input sources.
- More robust handling of invalid identifiers in PyBEL inputs in
`indra.sources.bel`.
- BioPAX input processor in `indra.sources.biopax` now eliminates spurious
duplicate extractions by patterns
- New analysis script is available in `indra.sources.trips` for TRIPS/CWMS
extractions.

Core assembly modules
- Belief engine makes a new Bayesian scorer class available in `indra.belief`
which allows integrating curation directly into belief scores.
- Preassembler in `indra.preassembler` preserves original grounding in addition
to the original entity texts upon Statement deduplication.
- The SiteMapper has been substantially refactored to take its core functionality from the `protmapper` package (https://github.com/indralab/protmapper).
- A new disambiguation tool, DEFT (https://github.com/indralab/deft) is now
integrated into the grounding mapping process in
`indra.preassembler.grounding_mapper`.
- Bug fixes and performance improvements in HierarchyManager in
`indra.preassembler.hierarchy_manager`.
- Bug fixes and performance improvements in OntologyMapper in
`indra.preassembler.ontology_mapper`.
- Assembly pipeline in `indra.tools.assemble_corpus` now supports merging
grounding from multiple evidences at the Statement level, as well as additional
preasssembly functionalities such as evidence flattening.

Tools
- New web service added for real-time curation of Statements with belief
updates, ontology extension, and re-grounding/re-assembly in
`indra.tools.live_curation`.

Output assemblers
- Major overhaul of the PySB Assembler in `indra.assemblers.pysb` to refactor
into multiple files, support individual statement-level policies, and passing
in parameterization from outside.
- CX assembler in `indra.assemblers.cx` supports setting custom network visual
styles.
- Major improvements and many new features in the HTML assembler for curation in
`indra.assemblers.html`, including grouping and ranking of Statements
- Fixes to the SBGN assembly format in `indra.assemblers.sbgn`.

Literature and database clients
- PMC client in `indra.literature.pmc` now supports getting text from NXML
- Protein lengths now available via the UniProt client in
`indra.databases.uniprot_client`

Resources
- All resource files updated to latest version as of 12/1/2018
- New ontology mapping resources for Hume and CWMS

REST API
- Multiple new input and assembly endpoints added to the REST API for Eidos,
CWMS, Sofia, and Hume, ontology mapping, and belief filtering.