Apsw-sqleet

=========

You can use the database as a `context manager
<http://docs.python.org/reference/datamodel.htmlwith-statement-context-managers>`_
as defined in :pep:`0343`. When you use *with* a transaction is
started. If the block finishes with an exception then the transaction
is rolled back, otherwise it is committed. See :meth:`Connection.__enter__`
for an example.

Behind the scenes the `savepoint
<https://sqlite.org/lang_savepoint.html>`_ functionality introduced
in SQLite 3.6.8 is used. Consequently :class:`Connection` *with*
blocks can be nested. If you use Connection level :ref:`execution
tracers <executiontracer>` then they will be called with the savepoint
SQL statements.

You can also use :class:`blobs <blob>` as a context manager which
ensures it is always closed when finished using it. See
:meth:`blob.__enter__` for an example.

Added :ref:`constants <sqliteconstants>`:

* SQLITE_SAVEPOINT (authorizer code)
* SQLITE_IOERR_CLOSE (extended result code)
* SQLITE_IOERR_DIR_CLOSE (extended result code)
* New mapping: SQLITE_FCNTL_LOCKSTATE, SQLITE_GET_LOCKPROXYFILE, SQLITE_SET_LOCKPROXYFILE, SQLITE_LAST_ERRNO. SQLite does not document the purpose of these except the first one.

Updated :ref:`vfs` test code. SQLite's routines that call
:meth:`VFSFile.xTruncate` used to ignore errors but now return an
error to the caller. :meth:`VFSFile.xFileControl` is now called so a
user implemented one must call any base it inherits from for SQLite to
function normally.

Updated the xDlSym VFS routine to have the different but compatible
type signature as changed in SQLite 3.6.7 to deal with pedantic
compiler warnings.

Fixed bug in :ref:`apswtrace <apswtrace>` that could result in poorly
formatted times. Leading comments are also stripped for queries
printed in the final reports. You can also request subsets of the
reports.

The :ref:`speedtest` script will now fallback to the Python builtin
sqlite3 module if it can't find an externally installed pysqlite.

==========

Windows binary download for Python 3.0 is :ref:`available
<source_and_binaries>`.

Various changes in data structures and containers to reduce code size.

Changed the code to handle SQLite errors to only use Python
functionality and no operating system functionality (thread local
storage). This also addresses :issue:`36` where Vista was not binary
compatible with XP. Thanks to Rudolf Gaertner for assistance in
detecting and diagnosing this issue.

:class:`Connections <Connection>`, :class:`cursors <Cursor>` and
:class:`blobs <blob>` can be used by `weak references
<http://docs.python.org/library/weakref.html>`_.

You can now install :class:`Connection` wide :meth:`execution
<Connection.setexectrace>` and :meth:`row <Connection.setrowtrace>`
:ref:`tracers <tracing>`.

The callbacks for execution and row tracers have a different signature
to include the cursor the execution or row happened on. This is a
backwards incompatible change. See :ref:`tracing <tracing>` for
details.

Due to popular demand, added :meth:`Cursor.fetchall`. This is a
longer way of typing ``list(cursor)``.

Added attributes to the :class:`Connection` class -
:attr:`~Connection.filename`, :attr:`~Connection.open_flags` and
:attr:`~Connection.open_vfs`. These let you track how the database
was opened.

Added a :ref:`apswtrace <apswtrace>` script to allow easy SQL tracing
without having to modify your code.

Revert to using older SQLite APIs in order to work around
:cvstrac:`2158`. (This also saves a little bit of SQLite memory
usage). The user visible effect was that you could get different
exceptions and error text depending on whether a query was already in
the :ref:`statement cache <statementcache>` or if you were
multi-threading. As an example, if you have a query that used an
unknown collation then SQLite's `prepare
<https://sqlite.org/c3ref/prepare.html>`_ returns
:const:`SQLITE_ERROR` with error text about the bad collation. If a
query had already been prepared, the collation removed and then `run
<https://sqlite.org/c3ref/step.html>`_ the new SQLite routines are
returning :const:`SQLITE_SCHEMA` and generic ``schema changed`` error
text. Changing user defined functions could also cause a previously
correct query to become invalid.

========

The distribution now includes a :ref:`speedtest` script. You can use
this to see how APSW performs relative to pysqlite, or to track
performance differences between SQLite versions. The underlying
queries are derived from `SQLite's speed test
<https://sqlite.org/src/finfo?name=tool/mkspeedsql.tcl>`_

The statement cache was completely rewritten. It uses less memory and
scales significantly better.

It was possible to get a deadlock between the Python GIL and the
SQLite database mutex when using the same :class:`Connection` across
multiple threads. Fixed by releasing the GIL in more places and added
test that inspects the source to verify GIL/mutex handling. Thanks to
amicitas reporting this as :issue:`31`

SQLite's API has been extended in 3.6.5 so that errors can be
retrieved in a thread safe manner. APSW now uses this API.

As a consequence of the prior two changes it is now possible and safe
to use the same :class:`Connection` across as many threads as you want
`concurrently <https://sqlite.org/threadsafe.html>`_.

Documentation is now done using `Sphinx <http://sphinx.pocoo.org>`_
which was adopted by Python 2.6 and 3. This has allowed for richer
documentation and more output formats such as PDF and `Windows CHM
<http://en.wikipedia.org/wiki/Microsoft_Compiled_HTML_Help>`_ format.

The binary distribution for Windows includes the `full text search
<https://sqlite.org/fts3.html>`__ (FTS) and `Rtree
<https://sqlite.org/src/finfo?name=ext/rtree/README>`_ extensions.
See also :ref:`setup_py_flags`.

The source structure and files were reorganized to make it clearer
where things are implemented and to make automatic extraction of
documentation easier.

========

You can now write your own :ref:`VFS` in Python. You can also inherit
from an existing VFS making it easy to augment or override small bits
of behaviour without having to code everything else. See the
:ref:`example <example-vfs>` where database files are obfuscated by
XORing their contents.

:file:`setup.py` now takes an optional :option:`--fetch-sqlite[=ver]`
argument to automatically download and use the latest SQLite
amalgamation (or a specified version). On non-Windows platforms it
will also work out what compile flags SQLite needs (for example
:const:`HAVE_USLEEP`, :const:`HAVE_LOCALTIME_R`). Several other
options to :file:`setup.py` are also available to control
enabling/omitting certains features and functionality. See
:ref:`building <Building>` for further details.

APSW checks that SQLite was compiled to be `threadsafe <https://sqlite.org/c3ref/threadsafe.html>`_

Added new constants:

* :const:`SQLITE_IOERR_ACCESS`, :const:`SQLITE_IOERR_CHECKRESERVEDLOCK` and :const:`SQLITE_IOERR_LOCK` extended result codes
* :const:`SQLITE_OPEN_NOMUTEX` and :const:`SQLITE_OPEN_FULLMUTEX` open flags
* Several new :const:`SQLITE_CONFIG` and :const:`SQLITE_STATUS` codes

Wrapped several new SQLite apis:

* `sqlite3_config <https://sqlite.org/c3ref/config.html>`_
* `sqlite3_initialize/sqlite3_shutdown <https://sqlite.org/c3ref/initialize.html>`_
* `sqlite3_memory_used/sqlite3_memory_highwater <https://sqlite.org/c3ref/memory_highwater.html>`_
* `sqlite3_status <https://sqlite.org/c3ref/status.html>`_
* `sqlite3_soft_heap_limit <https://sqlite.org/c3ref/soft_heap_limit.html>`_
* `sqlite3_release_memory <https://sqlite.org/c3ref/release_memory.html>`_
* `sqlite3_randomness <https://sqlite.org/c3ref/randomness.html>`_


The following experimental apis are not wrapped as there is nothing
useful you can do with them (yet):

* `sqlite3_db_config <https://sqlite.org/c3ref/db_config.html>`_
* `sqlite3_db_status <https://sqlite.org/c3ref/db_status.html>`_

Restored prior behaviour regarding Python ints and longs returning int
for numbers fitting in signed 32 bit. This only affects Python 2 as
Python 3 uses long exclusively. Thanks to Joe Pham for reporting this
as :issue:`24`

Added :meth:`Connection.getsqlite3pointer` method to help with
:issue:`26`

========

APSW now works with Python 3 (you need 3.0b1 or later).

(:issue:`17`)
Removed the :const:`SQLITE_MAX_*` constants since they could be
unreliable (eg APSW can't tell what a shared library was compiled
with). A workaround is documented in :func:`Connection.limit`.

========

APSW is now hosted at https://code.google.com/p/apsw

You can use this with SQLite 3.5.9 onwards.

SQLite now provides the source all `amalgamated
<https://sqlite.org/cvstrac/wiki?p=TheAmalgamation>`_ into one file
which improves performance and makes compilation and linking of SQLite
far easier. The build instructions are updated.

:const:`SQLITE_COPY` authorizer code and :const:`SQLITE_PROTOCOL`
error code are no longer used by SQLite, but the values are left in
apsw for backwards compatibility

:const:`SQLITE_IOERR_DELETE`, :const:`SQLITE_IOERR_BLOCKED` and :const:`SQLITE_IOERR_NOMEM`

:func:`Connection.interrupt` can be called from any thread

SQLite has implementation limits on string and blob lengths (roughly
constrained to fitting within a signed 32 bit integer - less than 2GB)
which weren't checked. Using a 64 bit Python 2.5+ (as I do) it would
have been possible to destroy memory and crash the
program. Consequently APSW has length checks to ensure it doesn't
happen. SQLite now has further `limits checking
<https://sqlite.org/limits.html>`_ which cover other things as well
such as maximum number of virtual machine opcodes, maximum number of
variables etc. These are very useful if you are taking in SQL
statements from elsewhere. Call :func:`Connection.limit`

A rename method was added for virtual tables.

SQLite 3.5 removed the requirement that all operations on a connection
be done in the same thread. Consequently all code that enforced the
restriction has been removed from APSW.

You no longer have to call :func:`Connection.close`. This was
previously a requirement to ensure that the correct thread was used
(destructors otherwise run in any thread). It is however still a good
idea to do so since you can catch exceptions when close is called
but not if you let the destructor do the closing.

SQLite now has incremental :ref:`blob I/O <blobio>`

:issue:`4` which could lead to generic error messages was fixed in
SQLite 3.5.9.

Fixed :issue:`1` error in example code for virtual tables which caused
filename errors on Windows.

Fixed :issue:`15` releasing the GIL around calls to sqlite3_prepare.

Fixed :issue:`7` ensuring that extension module filenames are
converted to utf8.

Use the `sqlite3_open_v2 <https://sqlite.org/c3ref/open.html>`_
interface which allows specifying which vfs to use. This release does
not allow you to write your own vfs as the SQLite vfs interface is
being changed for SQLite 3.6.

Used new SQLite functions that keep track of when virtual tables and
collations are no longer used so they can be released. Previously APSW
also had to keep track duplicating effort.

Improved test coverage a few more percent.

The statement cache now defaults to the same number of entries as
pysqlite (100). You can however specify more or less as needed.

:func:`Connection.collationneeded` was implemented.