Indra

**This is the last release of INDRA that is compatible with both Python 2.7 and Python 3.5+. The next release will drop Python 2 compatibility and maintain compatibility with Python 3.5+ only.**

Package structure, dependencies
- INDRA is now compatible with ndex2 version >= 2.0

Representation
- New `text_refs` attribute in Evidence which allows representing text
references other than PMID, e.g. DOI, URL, etc.
- Evidence str/repr display improved and extended to support pretty printing
large Evidence objects

Input processors
- Geographical location context extracted from Eidos in `indra.sources.eidos`
- EKB file-based processing added to TRIPS and CWMS APIs in
`indra.sources.trips` and `indra.sources.cwms`
- REACH processor now extracts and propagate entity coordinates from
REACH output in `indra.sources.reach`
- Sparser API has more robust timeout and failure handling, and multiple
improvements in the processor in `indra.sources.sparser`
- New function to submit curation via INDRA DB REST API in
`indra.sources.indra_db_rest`
- Improvements and extensions in INDRA DB REST API including the ability
to limit number of returned Statements in `indra.sources.indra_db_rest`
- Bug fixes in MedScan processor in `indra.sources.medscan`

Core assembly modules
- Belief engine refactored to pass evidences instead of scores when propagating
evidence on the refinements graph in `indra.belief`
- The assembly of large Complexes and Associations has been significantly
optimized in `indra.statements` and `indra.preassembler`

Output assemblers
- New HTML Statement browser and curation interface added in
`indra.assemblers.html`
- English assembler extended to handle Influence, Association, and Conversion
Statements in `indra.assemblers.english`
- Bug fixes in CX assembly and hub layout in `indra.assemblers.cx`

Literature and database clients
- New GO client added in `indra.databases.go_client` which allows normalizing
biological processes and cellular components to standardized names
- UniProt client can now retreive descriptions of proteins in
`indra.databases.uniprot_client`

Resources
- All resource files updated to latest version as of 10/30/2018
- Ontologies for SOFIA, Eidos and Hume, and mapping between them updated

Package structure
- The DB is no longer part of the INDRA package, and `indra_db_rest`, `indra.db` and `indra.tools.reading.db_reading` have been moved to https://github.com/indralab/indra_db
- Most of the models and applications in the `models` folder have been moved to https://github.com/sorgerlab/indra_apps with just some core examples and models remaining
- Files in the submodules in `indra.sources` are now following a standardized naming of `api.py` and `processor.py`. The corresponding APIs haven't changed.
- The `indra.assemblers` module is now divided into further submodules, one for each output assembler, e.g. `indra.assemblers.pysb`. This changes the way these modules need to be imported to e.g. `from indra.assemblers.pysb import PysbAssembler`.

Representation
- Object model for `Context` added, including `BioContext` and `WorldContext` and further classes. `Evidence` now has a `context` attribute to carry this information (which previously appeared in `annotations`).
- New `Association` Statement class to represent non-causal and non-directional relationships
- SBO annotations in JSON serialization are now obtional and off by default

Input processors
- New input processor and API for LINCS drug-target relationships (`indra.sources.lincs_drug`)
- New input processor and API for Target Affinity Spectrum data for drug-target relationships (`indra.sources.tas`)
- The Eidos input processor has numerous improvements and extenstions including producing `WorldContext` for each `Evidence`, carrying negation and hedging, and picking up more meta-information and annotations (`indra.sources.eidos.processor`)
- The Eidos reader has been updated to work with Eidos version 0.2.2 (`indra.sources.eidos.reader`)
- The BEL input processor now produces BioContext for each Evidence (`indra.sources.bel.processor`)
- Many new features in the INDRA DB REST client (`indra.sources.indra_db_rest`) including ordering, timeouts, various size limits, retries, etc.
- REACH input processor now produces BioContext for each Evidence (`indra.sources.reach.processor`)
- SIGNOR input processor now produces BioContext for each Evidence (`indra.sources.signor.processor`)
- SOFIA input processor now produces WorldContext for each Evidence (`indra.sources.sofia.processor`)
- TEES API and processor refactored to support batch reading (`indra.sources.tees`)


Core assembly modules
- The BeliefEngine in `indra.belief` has been refactored to allow subclassing the BeliefScorer to implement custom scoring approaches. The default scores have been moved to a resource file in `indra.resources`.
- The BeliefEngine can now handle negative as well as positive evidence for a given Statement.
- The Preassembler now adds annotations for raw texts corresponding to Agents that would otherwise be lost upon duplicate resolution.

Output assemblers
- New hub layout generator for CX networks in `indra.assemblers.cx`
- Improvements and extensions to English assembler in `indra.assemblers.english`

Literature and database client
- New client to the MeSH database to allow name lookup and standardization in `indra.databases.mesh_client`
- New client to the LINCS small molecule and protein tables in `indra.databases.lincs_client`
- NDEx client can now be used without explicitly passing credentials in `indra.databases.ndex_client`
- Improvements to Elsevier client to be able to extract text from a wider range of XML formats in `indra.literature.elsevier_client`

Resources
- Resource files for new sources and database clients added
- Statements JSON schema updated with Context object model

Setup and dependencies:
- INDRA now supports networkx 2.x
- The Cython version restriction is now removed, and the recommended installation method for pyjnius is through Github
- Extra dependencies for various modules are now given explicitly in setup.py
- Separate configuration module (`indra.config`) to manage `config.ini` and `db_config.ini` files
- Contribution guidelines added

Representation:
- Extensions to keys and hashes used to identify related Statements (`indra.statements`)
- Statements now implement a `contradicts` method to find contradictions

Input processors:
- Major improvements and adaptation to changes in Eidos (`indra.sources.eidos`), Hume (`indra.sources.hume`), CWMS (`indra.sources.cwms`), and SOFIA (`indra.sources.sofia`) reader inputs
- BBN reader renamed to Hume (`indra.sources.hume`)
- Ontologies for the Eidos, Hume, and CWMS readers are integrated, and grounding information from these sources is represented
- New ISI reader API and processor added (`indra.sources.isi`)
- New Medscan reader API and processor added (`indra.sources.medscan`)
- Generalized and refactored SIGNOR input API and processor added (`indra.sources.signor`)
- BEL resources are now PyBEL compatible and processing defaults to PyBEL, rather than BEL/RDF (`indra.sources.bel`)
- Imrpoved modification and mutation site extraction, and handling of complex controllers in REACH input processor (`indra.sources.reach`)
- Bug fixes and improvements in TEES (`indra.sources.tees`) and Sparser (`indra.sources.sparser`) inputs
- Broader set of Increase/DecreaseAmount Statements extracted from TRIPS (`indra.sources.trips`)
- Added robust DRUM reader to use TRIPS/DRUM locally (`indra.sources.trips.drum_reader`)

Core assembly modules
- Contradiction finding in Preassembler (`indra.preassembler`)
- Handling of new `is_opposite` relationship in Statements, HierarchyManager and Preassembler
- Generalized reasoning across `isa/partof` paths in HierarchyManager (`indra.preassembler.hierarchy_manager`)
- New Ontology Mapper added (`indra.preassembler.ontology_mapper`)
- SiteMapper uses cache to speed up sequence access (`indra.preassembler.site_mapper`)
- Assembly filters now extend across BoundConditions (`indra.tools.assemble_corpus`)

Output assemblers
- BMI wrapper implemented for PySB models for integrated simulation in workflows (`indra.assemblers.bmi_wrapper`)
- CAG Assembler can produce Jupyter-embeddable graphs and TSVs (`indra.assemblers.cag_assembler`)
- Many generalizations and improvements in PyBEL assembler (`indra.assemblers.pybel_assembler`)
- Bug fixes and improvements in PySB, SIF, Graph, English, IndexCard and CX assemblers

Literature and database clients:
- NewsAPI client added (`indra.literature.news_api`)
- Many new functionalities to access content in Elsevier client (`indra.literature.elsevier`)
- UniProt client (`indra.databases.uniprot_client`) extended with protein synonyms; optimizations in resource loading result in faster import times

Resources
- Many resource files updated, including FamPlex (`indra.resources.famplex`)
- New INDRA JSON schema and documentation added (`indra/resources/statements_schema.json`)

Analysis and explanation
- Paths graph is now a separate package, and a dependency of INDRA (https://github.com/johnbachman/paths_graph) and has been removed from `indra.explanation`

High-throughput reading and database
- REST API updates and extensions (`db_rest_api` and `rest_api`)
- Separate client and config modules for DB (`indra.db.client` and `indra.db.config`)
- New reading and preassembly managers for DB (`indra.db.reading_manager`, `indra.db.preassembly_manager`)
- New statistics module for DB (`indra.db.stats`)
- New reporting and log analysis functionalities for AWS-based reading (`indra.tools.reading` and `indra.tools.reading.db_reading`)
- Extensive improvements to running and managing AWS-based reading jobs (`indra.tools.reading`)

Configuration:
- INDRA now comes with a dedicated configuration file (created when
first importing indra) at ~/.config/indra/config.ini in which paths to various
resources and access keys can be configured

Representation:
- New Concept parent class for Agent. Concepts allow representing entities
of interest without the assumption of molecular state that Agents have.

Input processors
- BioGRID: new dedicated input processor for interactions from BioGRID
in `indra.sources.biogrid`. This is preferable to the BioGRID content
coming from Pathway Commons (via `indra.sources.biopax`), in some cases.
- CWMS: new API and processor for the TRIPS/CWMS reading system
(http://trips.ihmc.us/parser/cgi/cwmsreader) in `indra.sources.cwms`
- SOFIA: initial implementation for input processor for reading output from
the SOFIA reading system added in `indra.sources.sofia`
- Grounding information from Eidos, BBN and CWMS systems now represented and
used in assembly

Core assembly modules
- All core assembly procedures now extend to BoundConditions of Agents
- HierarchyManager generalized to allow merging multiple hierarchies
into a single manager
- New `isa` hierarchies added for non-biology use cases that now work with
HierarchyManager / Preassembler
- Custom grounding map can now be propagated into assembly from
`indra.tools.machine` and also via `indra.tools.assemble_corpus`

Output assemblers
- TSV Assembler: new assembler in `indra.assemblers.tsv_assembler` to
output knowledge in a tabular format, suitable for human curation

High-throughput reading and database
- REST API for database module to query INDRA Statement content from local or
remote database
- Various improvements and extensions of high-throughput reading framework
and database utilities

Representation:
- New Influence Statement to represent causal influence between a subject
and an object, and optionally the polarity and magnitude of change in
subject and object

Input processors
- TEES: new API in `indra.sources.tees` to read with the
Turku Event Extraction System
(https://github.com/jbjorne/TEES) and instantiate INDRA Statements
- PyBEL: new API in `indra.sources.bel` that allows processing PyBEL graphs
(https://github.com/pybel/pybel) as an input to INDRA
- NDEx CX: new API in `indra.sources.ndex_cx` to process networks on NDEx
(http://ndexbio.org) and extract INDRA Statements
- Signor: new API in `indra.sources.signor` to read the content of the
Signor signaling network database (https://signor.uniroma2.it/) as
INDRA Statements
- Geneways: new API in `indra.sources.geneways` to read the output generated
by the Genewayes reading system
(https://www.ncbi.nlm.nih.gov/pubmed/15016385)
- Eidos: new API in `indra.sources.eidos` to read text using the Eidos reading
system (https://github.com/clulab/eidos/) and process its output of
causal interactions into INDRA Statements
- BBN: new API in `indra.sources.bbn` to process causal interactions extracted
by BBN's world modelers reading system

Core assembly modules
- Flexible parameterization of the BeliefEngine in `indra.belief`, and support for prior
belief assignment based on more granular attributes of a Statement's evidence

Output assemblers
- Causal Analysis Graphs: new assembler in `indra.assemblers.cag_assembler`
to assemble into "Causal Analysis Graphs", and display with the Delphi package
(https://github.com/ml4ai/delphi)
- PyBEL: new assembler in `indra.assemblers.pybel_assembler` to assemble into
a PyBEL graph and allow analysis on PyBEL web
(https://pybel.scai.fraunhofer.de/)

Model analysis
- Integration with `kappy` to facilitate Kappa model analysis and simulation
- Several new, optimized graph data structures in `indra.explanation`

Tools
- The "machine" (basis of The RAS Machine) has been moved
to `indra.tools.machine`.

Database clients and resources
- FamPlex (renamed from Bioentities) is no longer a submodule, it is available
directly under `indra.resources.famplex`
- New ChEBML client in `indra.databases.chembl_client`
- New functionalities in cBio portal and ChEBI clients

High-throughput reading and database
- Various high-througput reading modules refactored into four submodules based
on use case in `indra.tools.reading`
- New database module in `indra.db` to cache INDRA Statements from
reading at scale

Package structure
- All knowledge sources have been moved to the `indra.sources` submodule

Representation:
- New Conversion Statement type for representing metabolic conversions
- RasGap/RasGef/RasGtpActivation renamed to the more general Gap/Gef/GtpActivation
- Improved JSON serialization with SBO mappings for INDRA Statements
- New graph visualization for Statements

Input processors:
- New input processor for NDEx CX networks in `indra.sources.ndex_cx`
- New API and input processor for Sparser (https://github.com/ddmcdonald/sparser)
- Better grounding prioritization and new extraction patterns in TRIPS processor

Core assembly modules:
- Better linking of modifications in MechLinker in `indra.mechlinker`
- More advanced automated SiteMapper mapping logic in `indra.preassembler.site_mapper`

Output assemblers
- Major improvements in PySB Assembler including
- new assembly policies (e.g. for Conversion)
- new rate laws (Michaelis-Menten and Hill function)
- better handing of grounded monomers
- extended semantic annotations referring to provenance and the role of
monomers in rules
- Assembler into KAMI in `indra.assemblers.kami_assembler`. KAMI is a knowledge
representation and meta-model for Kappa: https://github.com/Kappa-Dev/KAMI.
- Full reimplementation of SBGN assembler to follow reaction network-based assembly
- SifAssembler extended with new Loopy (http://ncase.me/loopy/) output mode
- Various improvements and extensions in CyJSAssembler and CxAssembler

Model Analysis
- New explanation module in `indra.explanation` to check INDRA-assembled PySB
models against a set of INDRA Statements

Tools
- New large-scale reading pipeline implemented for Amazon Batch
- New Amazon-based processing for the RAS Machine

Databases and Resources
- New client to cBio Portal web API for accessing patient and cell line data
- New client to PhosphoSite web API for accessing phosphosite locations and
mapping between organisms
- Extension of the Bioentities submodule with more extensive grounding entries
and references
- Mapping between human, rat and mouse proteins and genes in UniProt and HGNC clients
- Updates to all other database resource files

Other
- Extended REST API which can be run locally to interact with INDRA.
New API documentation are in the rest_api folder.
- New model projects and evaluations in `models`
- Extended documentation in `doc`