Backwards Compatibility Notes
* Support for Python 2.7 has been dropped. Python 3.5 is now the
minimum required Python version. (109)
* ``train_dictionary()`` now uses the ``fastcover`` training mechanism
(as opposed to ``cover``). Some parameter values that worked with the old
mechanism may not work with the new one. e.g. ``d`` must be ``6`` or ``8``
if it is defined.
* ``train_dictionary()`` now always calls
``ZDICT_optimizeTrainFromBuffer_fastCover()`` instead of different APIs
depending on which arguments were passed.
* The names of various Python modules have been changed. The C extension
is now built as ``zstandard.backend_c`` instead of ``zstd``. The
CFFI extension module is now built as ``zstandard._cffi`` instead of
``_zstd_cffi``. The CFFI backend is now ``zstandard.backend_cffi`` instead
of ``zstandard.cffi``.
* ``ZstdDecompressionReader.seekable()`` now returns ``False`` instead of
``True`` because not all seek operations are supported and some Python
code in the wild keys off this value to determine if ``seek()`` can be
called for all scenarios.
* ``ZstdDecompressionReader.seek()`` now raises ``OSError`` instead of
``ValueError`` when the seek cannot be fulfilled. (107)
* ``ZstdDecompressionReader.readline()`` and
``ZstdDecompressionReader.readlines()`` now accept an integer argument.
This makes them conform with the IO interface. The methods still raise
``io.UnsupportedOperation``.
* ``ZstdCompressionReader.__enter__`` and ``ZstdDecompressionReader.__enter__``
now raise ``ValueError`` if the instance was already closed.
* The deprecated ``overlap_size_log`` attribute on ``ZstdCompressionParameters``
instances has been removed. The ``overlap_log`` attribute should be used
instead.
* The deprecated ``overlap_size_log`` argument to ``ZstdCompressionParameters``
has been removed. The ``overlap_log`` argument should be used instead.
* The deprecated ``ldm_hash_every_log`` attribute on
``ZstdCompressionParameters`` instances has been removed. The
``ldm_hash_rate_log`` attribute should be used instead.
* The deprecated ``ldm_hash_every_log`` argument to
``ZstdCompressionParameters`` has been removed. The ``ldm_hash_rate_log``
argument should be used instead.
* The deprecated ``CompressionParameters`` type alias to
``ZstdCompressionParamaters`` has been removed. Use
``ZstdCompressionParameters``.
* The deprecated aliases ``ZstdCompressor.read_from()`` and
``ZstdDecompressor.read_from()`` have been removed. Use the corresponding
``read_to_iter()`` methods instead.
* The deprecated aliases ``ZstdCompressor.write_to()`` and
``ZstdDecompressor.write_to()`` have been removed. Use the corresponding
``stream_writer()`` methods instead.
* ``ZstdCompressor.copy_stream()``, ``ZstdCompressorIterator.__next__()``,
and ``ZstdDecompressor.copy_stream()`` now raise the original exception
on error calling the source stream's ``read()`` instead of raising
``ZstdError``. This only affects the C backend.
* ``ZstdDecompressionObj.flush()`` now returns ``bytes`` instead of
``None``. This makes it behave more similarly to ``flush()`` methods
for similar types in the Python standard library. (78)
* ``ZstdCompressionWriter.__exit__()`` now always calls ``close()``.
Previously, ``close()`` would not be called if the context manager
raised an exception. The old behavior was inconsistent with other
stream types in this package and with the behavior of Python's
standard library IO types. (86)
* Distribution metadata no longer lists ``cffi`` as an ``install_requires``
except when running on PyPy. Instead, ``cffi`` is listed as an
``extras_require``.
* ``ZstdCompressor.stream_reader()`` and ``ZstdDecompressor.stream_reader()``
now default to closing the source stream when the instance is itself
closed. To change this behavior, pass ``closefd=False``. (76)
* The ``CFFI`` backend now defines
``ZstdCompressor.multi_compress_to_buffer()`` and
``ZstdDecompressor.multi_decompress_to_buffer()``. However, they
raise ``NotImplementedError``, as they are not yet implemented.
* The ``CFFI`` backend now exposes the types ``ZstdCompressionChunker``,
``ZstdCompressionObj``, ``ZstdCompressionReader``,
``ZstdCompressionWriter``, ``ZstdDecompressionObj``,
``ZstdDecompressionReader``, and ``ZstdDecompressionWriter`` as
symbols on the ``zstandard`` module.
* The ``CFFI`` backend now exposes the types ``BufferSegment``,
``BufferSegments``, ``BufferWithSegments``, and
``BufferWithSegmentsCollection``. However, they are not implemented.
* ``ZstdCompressionWriter.flush()`` now calls ``flush()`` on the inner stream
if such a method exists. However, when ``close()`` itself calls
``self.flush()``, ``flush()`` is not called on the inner stream.
* ``ZstdDecompressionWriter.close()`` no longer calls ``flush()`` on
the inner stream. However, ``ZstdDecompressionWriter.flush()`` still
calls ``flush()`` on the inner stream.
* ``ZstdCompressor.stream_writer()`` and ``ZstdDecompressor.stream_writer()``
now have their ``write_return_read`` argument default to ``True``.
This brings the behavior of ``write()`` in compliance with the
``io.RawIOBase`` interface by default. The argument may be removed
in a future release.
* ``ZstdCompressionParameters`` no longer exposes a ``compression_strategy``
property. Its constructor no longer accepts a ``compression_strategy``
argument. Use the ``strategy`` property/argument instead.
Bug Fixes
* Fix a memory leak in ``stream_reader`` decompressor when reader is closed
before reading everything. (Patch by Pierre Fersing.)
* The C backend now properly checks for errors after calling IO methods
on inner streams in various methods. ``ZstdCompressionWriter.write()``
now catches exceptions when calling the inner stream's ``write()``.
``ZstdCompressionWriter.flush()`` on inner stream's ``write()``.
``ZstdCompressor.copy_stream()`` on dest stream's ``write()``.
``ZstdDecompressionWriter.write()`` on inner stream's ``write()``.
``ZstdDecompressor.copy_stream()`` on dest stream's ``write()``. (102)
Changes
* Bundled zstandard library upgraded from 1.4.5 to 1.4.8.
* The bundled zstandard library is now using the single C source file
distribution. The 2 main header files are still present, as these are
needed by CFFI to generate the CFFI bindings.
* ``PyBuffer`` instances are no longer checked to be C contiguous and
have a single dimension. The former was redundant with what
``PyArg_ParseTuple()`` already did and the latter is not necessary
in practice because very few extension modules create buffers with
more than 1 dimension. (124)
* Added Python typing stub file for the ``zstandard`` module. (120)
* The ``make_cffi.py`` script should now respect the ``CC`` environment
variable for locating the compiler. (103)
* CI now properly uses the ``cffi`` backend when running all tests.
* ``train_dictionary()`` has been rewritten to use the ``fastcover`` APIs
and to consistently call ``ZDICT_optimizeTrainFromBuffer_fastCover()``
instead of different C APIs depending on what arguments were passed.
The function also now accepts arguments ``f``, ``split_point``, and
``accel``, which are parameters unique to ``fastcover``.
* CI now tests and builds wheels for Python 3.9.
* ``zstd.c`` file renamed to ``c-ext/backend_c.c``.
* All built/installed Python modules are now in the ``zstandard``
package. Previously, there were modules in other packages. (115)
* C source code is now automatically formatted with ``clang-format``.
* ``ZstdCompressor.stream_writer()``, ``ZstdCompressor.stream_reader()``,
``ZstdDecompressor.stream_writer()``, and
``ZstdDecompressor.stream_reader()`` now accept a ``closefd``
argument to control whether the underlying stream should be closed
when the ``ZstdCompressionWriter``, ``ZstdCompressReader``,
``ZstdDecompressionWriter``, or ``ZstdDecompressionReader`` is closed.
(76)
* There is now a ``zstandard.open()`` function for returning a file
object with zstd (de)compression. (64)
* The ``zstandard`` module now exposes a ``backend_features``
attribute containing a set of strings denoting optional features
present in that backend. This can be used to sniff feature support
by performing a string lookup instead of sniffing for API presence
or behavior.
* Python docstrings have been moved from the C backend to the CFFI
backend. Sphinx docs have been updated to generate API documentation
via the CFFI backend. Documentation for Python APIs is now fully
defined via Python docstrings instead of spread across Sphinx ReST
files and source code.
* ``ZstdCompressionParameters`` now exposes a ``strategy`` property.
* There are now ``compress()`` and ``decompress()`` convenience functions
on the ``zstandard`` module. These are simply wrappers around the
corresponding APIs on ``ZstdCompressor`` and ``ZstdDecompressor``.