Changelogs » Concrete

PyUp Safety actively tracks 268,179 Python packages for vulnerabilities and notifies you when to upgrade.



  - Nothing changed yet.


  - Fixed bug where `concrete.context` code was not included in 4.15.0


* Added `Context` structure and `AnnotateWithContextService` service.
  `Context`s are intended to be used to convey additional information
  alongside a `Communication` to be annotated.


  - Updated to Concrete schema 4.15, which adds `Context` structure and
  `AnnotateWithContextService` service.  `Context`s are intended to be
  used to convey additional information alongside a `Communication` to
  be annotated.
  - Dropped support for Python 2.7
  - Added `add_references` flag to CommunicationContainer classes
  - Added shell script ``
  for installing accelerated Thrift on macOS using the Homebrew
  package manager.


  - Added `scripts/` and `scripts/` script for
  converting Communication archives to/from a CSV file of TJSONProtocol
  encoded Communications
  - Updated Thrift dependency from 0.10.0 to 0.11.0
  - Added `--uri` flag for THttp/TJSONProtocol support to
  `scripts/` and
  - Added Concrete Service wrappers using Thrift THttp/TJSONProtocol
  - CommunicationReader.__next__() now throws EOFError when reading a
  truncated (invalid) Communication file.
  - Improvements to `scripts/`
  - Command line argument for hostname changed from positional
  argument to optional `--host` flag
  - Command line argument for port changed from positional
  argument to optional `--port` flag
  - Added batch mode for command line queries
  - Search terms with Non-ASCII characters now properly supported on
  both Python 2 and Python 3
  - Added `--with-scores` flag to print search result scores
  - Improved newline handling for concrete.util.simple_comm.create_comm()
  - Leading and trailing document newlines no longer generate empty
  - Multiple lines of whitespace are now treated as section breaks;
  previous behavior was that only '\n\n' was used for section
  - Twitter (sometimes) uses the incorrect ISO-639-1 code for
  Indonesian.  The code should be 'id', but Twitter sometimes uses
  'in'.  `concrete.util.twitter.twitter_lid_to_iso639_3()` now
  converts this incorrect ISO-639-1 code to the correct ISO-639-3 code
  for Indonesian ('ind').
  - Fixed import errors in `examples/`
  caused by schema changes


* Removed `constituent` field from `MentionArgument` and
  * Added `dependencies` and `constituent` fields to `TokenRefSequence`


  - Updated for Concrete schema 4.14
  - Removed `constituent` field from `MentionArgument` and
  - Added `dependencies` and `constituent` fields to `TokenRefSequence`


  - Fix bug in in which timeout would sometimes
  be ignored.


  - Use home as WORKDIR in Docker image to avoid import errors.
  - Add .dockerignore.
  - Add script to wait for a Concrete service to become alive.
  - Add S3 fetch and store server scripts.


  - Added 'bottle' dependency needed by scripts/
  - Add S3 fetch (communication container) and store handler.
  - Add Redis store handler.
  - Add concrete.util.summarization_wrapper submodule for
  SummarizationService introduced with concrete 4.13


* Added the `summarization` package, which includes the enumeration
  `SummarySourceType`, the structs `SummarizationRequest`,
  `SummaryConcept`, `Summary`, and `SummarizationCapability`, and the
  service `SummarizationService`.
  * Added an `id` field to the class `Entity`.
  * Added a `rawMentionList` field to the class `Entity`.
  * Added an `entity` field to `SearchResultItem`.
  * Added `TOPICID` to the `AnnotationTaskType` enumeration


  - Update to concrete 4.13 schema.


  - Breaking change: update to support multiple
  annotations of the same type.  Changes CoNLL output for an annotation
  type that is not present: an empty column is no longer printed.
  - Refactor concrete.inspect to use annotation filter callbacks to select
  annotations rather than tool names, as tool names no longer uniquely
  identify an annotation.  The new filter_annotations function can be
  used to create simple filters; that functionality is exposed in
  - Add timestamp_to_datetime in concrete.util.metadata
  - Add optional gzip compression to CommunicationWriter
  - Print situation mention kind and intensity in concrete.inspect


  - Clean up sphinx doc: suppress thrift_spec fields, render
  comments from Thrift verbatim
  - Update help for --annotation-headers
  (and a few other help messages)
  - Add --other-tag option to to enable CoNLL output
  of arbitrary token tags
  - Move get_tokenizations from concrete.inspect to
  - Added PropertyLists to situation-mentions output of
  - Fix concrete2json TJSONProtocol UTF-8 bug


  - Store tweet JSON in Communication.originalText.
  - Expand/modify functionality of
  - Accept communication IDs on standard input
  - Write results to tar.gz archive
  - Expand functionality of
  - Accept search terms as command-line arguments
  - Populate SearchQuery.rawQuery field
  - Add command-line flags --about, --alive, --capabilities, --corpora
  - Add child mentions to " --entities" output.
  - concrete.util.simple_comm.create_sentence now sets Token.textSpan field
  - CommunicationWriterTar.write() now uses CommID for default filename
  - concrete.validate.validate_communication_file() now returns a boolean
  - Added script
  - Now using Sphinx to generate documentation
  - can now read Communication IDs from stdin,
  and save Communications to a TGZ file


  - Add brief tutorial to README.rst.
  - Switch to zest.releaser for releases.
  - Add Python 3 support.


  - Refactor build, simplifying with tox.
  - Turn on public Appveyor and GitHub builds.
  - Remove hard numpy/scipy dependency in concrete.util.tokenization,
  adding (slower) fallback implementations.
  - Add to top-level package import.
  - Expand Windows test coverage to (most) integration tests.
  - Update package-level imports and use them in README.rst (as before).
  - Rename to
  - Add Redis hash backed Communication container implementation.


  - Add command-line Fetch client, server scripts.
  - Add wrappers for all current Concrete (4.12) services.
  - Add Communication container interface and implementations:
  - filesystem directory
  - Fetch service
  - file loaded into memory
  - zip file
  - Increase flake8 max-line-length from 79 to 99.
  - Add note about accelerated protocol C++ dependency to README.rst.
  - Fix syntax error in CONTRIBUTING.rst.


  - Deprecate remaining scripts containing underscores (to be removed in
  the future):
  - Add new scripts using hyphens instead of underscores, for easy typing:
  - Fix import errors in new AnnotateCommunicationService scripts:
  - examples/
  - scripts/
  - Add check for accelerated protocol.  This can be used directly as
  concrete.util.thrift_factory.is_accelerated(); acceleration is also
  now reported in the version string reported by scripts (when the
  --version flag is passed).


  - Select accelerated compact protocol in thrift_factory.  The
  accelerated protocol will now be used by default; if it is not
  supported, Thrift will fall back to the unaccelerated protocol.
  - Switch to reStructuredText in README and CONTRIBUTING to improve look
  and feel on PyPI.


  - Update to Thrift 0.10.0.
  - __hash__ implementations are removed for mutable objects,
  including all Concrete types.  This is in response to the existing
  __hash__ implementation breaking on lists and other mutable
  container types.
  - Remove accel branch from documentation and build.  Thrift 0.10.0
  includes the accelerated compact protocol, so the accelerated
  protocol is now available on Concrete-Python master and installations
  using pip.
  - The accel branch is now defunct; 4.12.1 is the final release on
  the accel branch.
  - Add g++ to Docker image, needed to build accelerated protocols.
  - Rename build.bash to generate.bash (affects maintainers only).


  - accel branch corresponding to 4.12.0.


* Merged `concrete-services` with the main repository.
  * Added a `k` field to the class `SearchQuery`.
  * Added a `communication` field to the class `SearchQuery`.


  - Update to concrete 4.12 schema.
  - Remove wrappers and scripts interfacing with the Annotator service
  removed in the 4.12 schema:
  - concrete.util.annotator_wrapper.AnnotatorClientWrapper
  - concrete.util.annotator_wrapper.AnnotatorServiceWrapper
  - concrete.util.annotator_wrapper.SubprocessAnnotatorServiceWrapper
  - examples/
  - examples/
  - scripts/
  - Add wrappers and scripts interfacing with the
  AnnotateCommunicationService (replacing the Annotator service):
  - concrete.util.annotate_wrapper.AnnotateCommunicationClientWrapper
  - concrete.util.annotate_wrapper.AnnotateCommunicationServiceWrapper
  - concrete.util.annotate_wrapper.SubprocessAnnotateCommunicationServiceWrapper
  - examples/
  - examples/
  - scripts/
  - Rename to
  - Remove deprecated compress-uuids, tweets2concrete scripts.


* Added retweeted status fields to the `TwitterInfo` structure,
  mirroring the fields representing replies.


The `4.10` release removes the `Sender` service and deprecates
  the `Annotator` service. These new services now live in a project
  called `concrete-services`, a parallel set of thrift files
  based on `concrete`. See that project for more details.


The `4.9` release contains an additional service, `Sender`, that
  allows `Communication` objects to be sent from clients to an
  implementing server.
  See the stub [here](thrift/services.thriftL66).


* Added service layer to Concrete. The .thrift file can be
  viewed [here](thrift/services.thrift).


* Added section-level `LanguageIdentification` field
  to allow language-specific analytics to run over meaningful


* Added new fields to [email.thrift](thrift/email.thrift) to better
  support/describe email messages.
  * Added a new type, [SpanLink](thrift/structure.thriftL457), to
  support linking between documents (e.g., mapping URLs within a
  * Added documentation about how to support pro-drop.
  * Fixed a few documentation issues.


Added an optional field, [ConstituentRef](thrift/structure.thriftL91), to both
  `MentionArgument`s and `SituationMention`s. This field allows references to
  Constituent objects within a Parse object.


Added an optional field, mentionSetId, to EntitySet. This field stores
  the UUID of an EntityMentionSet.
  When present, this field indicates that all Entities in the EntitySet
  contain mentions in the referenced EntityMentionSet.