Arctic

🍎 New Platforms

- This release coincides with the release of 1.3.0 on MacOS Apple Silicon on conda-forge!

🚀 Features

- Multiple improvements to the reliability and stability of the symbol list caching logic. (393)
- Add get_uri function to the Arctic instance (433)

🐛 Fixes

- Significant performance improvements for `read_batch`, `read_metadata_batch` and `get_description_batch` (415 435)
- Note that the performance improvements for `read_batch` will largely only be visible if no `QueryBuilder` query is passed in.
- `fast_tombstone_all` option has been removed - in code this is assumed to always be true. (366)
- Simplify log config and fix 426 453 (474)


<details>
<summary>Uncategorized</summary>

- rename isin test to avoid being skipped (418)
- github: Adapt bug report template (429)
- github: switching action for micromamba (424)
- 146 Add Black formatting to build (431)
- Higher utility GitHub Actions changes (438)
- fix black formatting issues previous PR (444)
- dev: Add simple `pre-commit` setup (445)
- build: Do not build with Remotery by default (466)
- follow-up: dev: Add simple `pre-commit` setup (448)
- cmake: Improve the resolution of LZ4 (451)
- fix: Required adaptations for MSVC 14.29.30133 support (459)

</details>

---
> The wheels are on [Pypi](https://pypi.org/project/arcticdb/). Below are for debugging:

This is a bugfix release:

- 419 409 Build fixes to enable the Conda build (and therefore publishing to conda-forge)
- 388 ArcticDB will now allow updates on data if the existing data was written prior to v1.1 regardless of whether that existing data is sorted or not

**Full Changelog**: https://github.com/man-group/ArcticDB/compare/v1.2.0...v1.2.1
---

ArcticDB distributes:
- wheels for Windows and Linux on [PyPI](https://pypi.org/project/arcticdb/), the latest being installable with:
bash
pip install --update arcticdb

- conda packages for MacOS and Linux on [conda-forge](https://anaconda.org/conda-forge/arcticdb), the latest being installable with:
bash
mamba install -c conda-forge --update arcticdb

Below are artifacts for debugging:

- Add support for snapshotting specific `symbol` + `version` pairs

New Platforms
:white_check_mark: Version 1.1 includes the first official release of the Windows version of ArcticDB! <img src="https://upload.wikimedia.org/wikipedia/commons/4/44/Microsoft_logo.svg" width="16" />
Versions 1.1 includes the following caveats:
* Please have a recent [Visual C++ Redistributable for Visual Studio 2015-2022](https://aka.ms/vs/17/release/vc_redist.x64.exe) installed.
* Windows does not support fixed-width NumPy strings and so will not be able to write fixed-width strings contained within a NumPy array.
* Writing pickled data can fail on machines with limited memory, see 340 for details and a workaround.
* The LMDB store on Windows pre-allocates the max storage size on disk, so we limited the size to 128MB. We will provide an option to increase the size in the future (229).

:white_check_mark: Version 1.1 now supports Python 3.11!

New Features
:white_check_mark: batch_get_descriptor method added (219)
:white_check_mark: batch_get_metadata method added (220)
Both batch methods do not achieve optimal parallelism - we are aiming to address in 1.2.

Changes
:white_check_mark: 1.1 pins to Pandas < 2.0 whilst we address a few remaining compatibility issues :panda_face: (237)
:white_check_mark: 1.1 supports Protobuf V3 as well as V4
:white_check_mark: Out-of-order data updates now raises exceptions should they prevent future indexed reads (203)
:white_check_mark: PCRE is now statically linked (321)
:white_check_mark: `stream id` which is a confusing term has been renamed to `symbol` in error messages and should be better understood (311)
:white_check_mark: Pickled data supports up-to 4GB (260)
:white_check_mark: Debugging toolbox added (209)
:white_check_mark: Introduced a base exception type `arcticdb.exceptions.ArcticException`. Exceptions are exposed in `arcticdb.exceptions` module with the following hierarchy:


RuntimeError
└-- ArcticException
|-- ArcticNativeNotYetImplemented
|-- DuplicateKeyException
|-- MissingDataException
|-- NoDataFoundException
|-- NoSuchVersionException
|-- NormalizationException
|-- PermissionException
|-- SchemaException
|-- SortingException
| └-- UnsortedDataException
|-- StorageException
|-- StreamDescriptorMismatch
└-- InternalException


Next version:

NEW FEATURE: defragment_symbol_data method added (180)

> The wheels are on [Pypi](https://pypi.org/project/arcticdb/). Below are for debugging: