Genomicsqlite

* Fix a concurrency bug introduced by fixing the prior concurrency bug in v0.9.3
* Update SQLite & Zstandard versions

<hr/>

**Start here** 👉 [](https://mlin.github.io/GenomicSQLite/) 👈

[](https://pypi.org/project/genomicsqlite/) [](https://anaconda.org/mlin/genomicsqlite) [](https://crates.io/crates/genomicsqlite)

Most developers should install a package through pip/conda/maven/cargo. They bundle the same prebuilt library files available below; the latter can be used with C/C++ projects and yet-to-be-packaged languages.

* **libgenomicsqlite.so** for Linux x86-64 (Ivy Bridge), built in CentOS 7
* **libgenomicsqlite.dylib** for macOS Catalina x86-64

An up-to-date version of SQLite itself (&geq;3.31.0) is also required, as detailed in the [docs](https://mlin.github.io/GenomicSQLite/). If unable to upgrade the host SQLite, a prebuilt Linux x86-64 `libsqlite3.so.0` is available below, for deployment with suitable `LD_LIBRARY_PATH`.

**USE AT YOUR OWN RISK:** This project is not associated with the SQLite developers.

<hr/>

_{SHA-256 digests:}


baa1440514341f2f5327b66fa8d94d4752b7bfab4a27e266c3cd9d6377310f79 libgenomicsqlite.so
4be1a68c6894664bb88c56aedd1b63c31d846c9d6d1eb7e1dd69c40510494b88 libgenomicsqlite.dylib
4485b89bfcac56ae20dd0ecfbca07f6a6cd53f59f18eb3a31ce6cc4d2f322950 genomicsqlite-jdbc-v0.9.4.jar
56d8684bd694cecbd1961cb690c360c8d02e03665a2e7d7ec780ddda47821367 libsqlite3.so.0

* Fix a concurrency bug potentially causing corruption in a multithreaded process writing multiple database files simultaneously.
* Notice: the newest SQLite versions as of this writing, v3.40.0 and v3.40.1, have a [query planning regression](https://sqlite.org/forum/forumpost/a19eef4e55) moderately impacting GRI query performance. A fix is expected in SQLite's next minor version. In the meantime, affected projects should pin SQLite v3.39.4 if possible.

<hr/>

_{SHA-256 digests:}


b57cecf021473959733c5c8b699967d320baca62f80b87e2575cb6014695c768 libgenomicsqlite.so
5a900bbecf5e7add3d6c60d3a5fec000f072de5a5e9328ddd79c107168b357e0 libgenomicsqlite.dylib
bbe1501f12a691ae813397dc190defd9a40f4a6c3fa7baa76ff3d5adc6eb6deb genomicsqlite-jdbc-v0.9.3.jar
09b5f5b30785413789375cd51c76bdb6131ea3ac5982e4c5959c84329f3cab9b libsqlite3.so.0

* Set `pragma secure_delete = ON` automatically, to ensure that deleting data shrinks the compressed file commensurately
* Bump zstd_vfs and SQLite versions

<hr/>

_{SHA-256 digests:}


6386d4e319e3e6b744295e12343ca0e8f0a33439a7955e490b5994e84335ad1a libgenomicsqlite.so
0778ac735a850f68c58c071602394fabfb805d7f6d3476e6eae49bf45eec31cf libgenomicsqlite.dylib
f9fdec286995fd82cf9383739cf2a7d02287e586bc4de4547172772a06f2df34 genomicsqlite-jdbc-v0.9.2.jar
d992d938246ca4b8ee3f242578defcb5d80c40f097e9e4f418d7c33fcd73ffc6 libsqlite3.so.0

* **Simplified usage of `genomic_range_index_levels()`**<br/>
The `genomic_range_index_levels()` helper for the [GRI level bounds optimization](https://mlin.github.io/GenomicSQLite/guide_gri/#level-bounds-optimization) now caches its analysis across subsequent invocations on the same SQLite connection (so long as the database isn't modified in the meantime). This means the in-SQL helper can be inlined into queries, instead of the driver program invoking it separately beforehand, in most cases. (If querying one item at a time, the latter approach would still be slightly more efficient.)

* **Streamlined web access with .dbi helper files** *(experimental)*<br/>
Access to compressed databases over HTTP[S] can now be made more efficient with a [small .dbi helper file](https://mlin.github.io/GenomicSQLite/guide_db/#web-access-optimization-with-dbi-helper-files) served alongside. The `genomicsqlite` command-line utility can generate the .dbi helper for a given read-only database, while the client automatically probes for it by appending `.dbi` to the main URL. <br/>
The .dbi is a compilation of key index pages from the main database file. Using this is optional, but often helpful for big databases accessed with high-latency requests. Without, the client might need to issue many small requests to collect those pages.

* **Bumped zstd, SQLite, rust dependencies**

<hr/>

_{SHA-256 digests:}


e0b15a0e5a7c5f65f754fe5cf7cfcb581a122d70f741d9d27af13c9d6dbf60cc libgenomicsqlite.so
f910fc659bbd5fa393bd251f807348eb1a7b4e96535fc509efc9e7267e4fed65 libgenomicsqlite.dylib
4a226c92c5fa565230e2ad693895464b4e1b2cf8b60a1e7d94c678b8028038c0 genomicsqlite-jdbc-v0.9.1.jar
8b4c0c422f9ad8a61b5ff42603bd340259b6029450e90265e7a5cfc5d66ede87 libsqlite3.so.0

*Withdrawn*

Improves efficiency of prefetching logic for web access to database files

<hr/>

_{SHA-256 digests:}


b7403da1741c01f068ff2f38ecce908a2b4a9034dbfce726e2114e1f234e5f64 libgenomicsqlite.dylib
66ca6e20936fbffadb4397743b96db57bd21af59a618d8ca9be033916a1f3a5f libgenomicsqlite.so
82634336f07b948d74b539a741f3fd9a9a3222c41955c86711ebb596b0621716 genomicsqlite-jdbc-v0.8.1.jar
2dae78da03f2dd1bdeac5089cf4ec976912efab23186e5b6c0c8ce91048d16b9 libsqlite3.so.0