Kerko

Features:

- Add the "Publication year" search scope. It is enabled by default, but can be
disabled by setting the `kerko.scopes.pubyear.enabled` configuration parameter
to `false`.
- Add new configuration parameter `kerko.pages.*.` for defining content pages
whose content come from selected Zotero standalone notes.
- Add a `"page"` type for the `kerko.link_groups.*.` configuration table, to
allow the creation of links to content pages.
- Add configuration parameters `kerko.performance.whoosh_index_memory_limit` and
`kerko.performance.whoosh_index_processors` to give some control over the
Whoosh search engine's indexing performance.
- Allow italic, bold, subscript, superscript and small-caps on Zotero fields
(such as titles and abstracts) where [rich text formatting
tags](https://www.zotero.org/support/kb/rich_text_bibliography) are used.
- Add Spanish translation. Thanks to [Albert
Ormazabal](https://github.com/aormazabal).

Bug fixes:

- Fix item title missing from item pages and from Atom feeds when item type is
Case, Email, or Statute.
- Fix incorrect sorting of search results by title when item type is Case,
Email, or Statute.
- Fix incorrect escaping of HTML markup in the HTML content elements of Atom
feeds.
- Fix incorrect scope, analyzer, and boost factor associated with the `caseName`
search field.
- Fix crash when a facet specifies no sort order.
- Fix crash on empty search result pages when configuration parameter
`kerko.features.print_results_max_count` is greater than zero.
- Fix badges displayed too close to title on item pages.

Other changes:

- Add `rel="noopener"` to `target="_blank"` links.
- Add `rel="noreferrer"` to links derived from Zotero library data, e.g.,
user-provided URLs, DOIs, link attachments.
- Reduce the default memory limit for Whoosh's index writer from 256 MB to 128
MB. This can prevent swapping with large libraries on small machines. The
default limit may now be changed with the
`kerko.performance.whoosh_index_memory_limit` parameter.
- Improve speed of single-item search pages, i.e., search pages having the
`page-len=1` URL parameter. This is achieved through removal of the `id`
parameter from pagination URLs. Obtaining those ids was requiring extra
processing. Instead, the `id` parameter is now added through the browser
History API when a pagination link is visited.
- Add support for Python 3.12.
- Replace pylint, pycodestyle, pydocstyle with Ruff.
- Replace Yapf with Ruff formatter. Reformat whole code base.
- Add pre-commit hooks. Run Ruff and other code checks on pre-commit.
- Add template blocks to facilitate theming.
- Improve documentation.

Backwards incompatible changes:

- Drop support for Python 3.7 (EOL).

Possibly backwards incompatible changes (more or less internal API changes):

- Kerko no longer provides a `kerko.blueprint` global object. Applications must
now instantiate the blueprint by calling `kerko.make_blueprint()` before
registering it. This prevents attempts to mutate the object after its
registration, especially in tests.
- Upgrade many dependencies, including new major versions of Flask (3.x),
Flask-Babel (4.x), Werkzeug (3.x).
- `kerko.extractors.TransformerExtractor` no longer applies transformers on
`None` values. If you need a transformer to process a `None` value (Kerko
itself has no such transformer), you now have to set the new `skip_none_value`
argument to `False` when instantiating the extractor.

Features:

- Add new configuration parameter `kerko.features.download_item`.

Bug fixes:

- Fix Gunicorn not exiting if the default TOML config has errors (use the
`errno.EINTR` error code). Not an issue anyone should have encountered!

Backwards incompatible changes:

- Rename the following configuration parameters:
- `kerko.features.download_results_link` → `kerko.features.download_results`.
- `kerko.features.print_item_link` → `kerko.features.print_item`
- `kerko.features.print_results_link` → `kerko.features.print_results`

Other changes:

- Improve documentation.

Features:

- Add new configuration parameters `kerko.link_groups.*.` for defining sets of
links that may be used for navigation.
- Add a breadcrumb trail to all pages. One or more base links can be configured
(see configuration parameters `kerko.breadcrumb` and
`kerko.link_groups.breadcrumb_base`).

Bug fixes:

- Fix validation pattern too restrictive for the `kerko.zotero.csl_style`
configuration parameter.

Backwards incompatible changes:

- Items from the main navigation bar are now based on the
`kerko.link_groups.navbar` configuration. There is no default translation for
it. To change the default text or to translate it, you must override the
default `kerko.link_groups.navbar` in your configuration file. If you have
overridden the `_navbar_items.html.jinja2` template and wish to use the
configuration, you might want to adapt it, or revert to the default template.

Other changes:

- Improve documentation.

Features:

- Add the `--full` option to the `sync` command line interface (CLI) command.

Bug fixes:

- Fix validation pattern too restrictive for the `kerko.facets.*.collection_key`
configuration parameter.
- Fix incorrect typing for the `kerko.features.download_results_max_count` and
`kerko.features.print_results_max_count` configuration parameters.

Other changes:

- Rename the default branch of the repository from 'master' to 'main'. If you
have cloned the repository with Git, use the following commands to rename your
local branch:

bash
git branch -m master main
git fetch origin
git branch -u origin/main main
git remote set-head origin -a


- Make the `config` CLI command show the configuration in TOML format.
- Make the `config` CLI command hide secrets by default. Add a `--show-secrets`
option.
- Improve documentation.

Backwards incompatible changes:

- Prevent the `clean` CLI command from cleaning everything by default. To delete
everything, you now have to run `flask kerko clean everything`. This may help
prevent accidental deletion of the cache, which in some instances can take
long to rebuild.
- Rename the following configuration parameters:
- `kerko.citation_formats.*` → `kerko.bib_formats.*`.
- `kerko.features.download_citations_link` → `kerko.features.download_results_link`
- `kerko.features.download_citations_max_count` → `kerko.features.download_results_max_count`
- `kerko.features.print_citations_link` → `kerko.features.print_results_link`
- `kerko.features.print_citations_max_count` → `kerko.features.print_results_max_count`
- Rename the following views:
- `item_citation_download` → `item_bib_download`
- `search_citation_download` → `search_bib_download`

*Warning:* Upgrading from version 0.9 or earlier will require that you adapt
your installation and configuration files, then rebuild your search index. Use
the commands below, then restart the application. If you are using KerkoApp,
make sure to check its [changelog][KerkoApp_changelog].

bash
flask kerko clean index
flask kerko sync index


Features:

- Add many new configuration parameters. Please refer to the configuration
parameters documentation for the full list.
- Add optional "Open in Zotero" and "View on zotero.org" buttons to item pages.
These are disabled by default (see new parameters
`kerko.features.open_in_zotero_app` and `kerko.features.open_in_zotero_web`).
Even when these are enabled, a user who wishes to use such button must first
enable it from the (also new) Preferences dialog.
- Add a web API endpoint for retrieving information about the last
synchronization from Zotero.
- Add the `kerko config` command to the Flask command line interface for
displaying all configuration parameters.

Other changes:

- Restructure and expand documentation into a unified documentation site for
both Kerko and KerkoApp.
- Add Portuguese translation. Thanks to Gonçalo Cordeiro.

Backwards incompatible changes:

- Raise the minimum required versions of Flask, Flask-Babel, Bootstrap-Flask,
and WTForms. If you have a custom application, some of those may introduce
breaking changes.
- The data directory has a new default location relative to the instance path.
Please check the documentation for the `DATA_PATH` and `INSTANCE_PATH`
configuration parameters. You may need to set one or both of those parameters,
and/or move your existing data directory.
- Almost all configuration parameters have been renamed and/or moved into a
hierarchical structure. Hierarchical parameters are referred to using
path-like, dot-separated parameter names, and may conveniently be set with the
`kerko.config_helpers.config_set()` function. Here is a mapping of the changed
parameters:
- `KERKO_BOOTSTRAP_VERSION` → `kerko.assets.bootstrap_version`
- `KERKO_CSL_STYLE` → `kerko.zotero.csl_style`
- `KERKO_COMPOSER` → `kerko_composer`
- `KERKO_DATA_DIR` → `DATA_PATH`. Now optional, relative to the instance
path, and defaulting to `kerko` instead of `data/kerko`.
- `KERKO_DOWNLOAD_ATTACHMENT_NEW_WINDOW` → `kerko.features.download_attachment_new_window`
- `KERKO_DOWNLOAD_CITATIONS_LINK` → `kerko.features.download_citations_link`
- `KERKO_DOWNLOAD_CITATIONS_MAX_COUNT` → `kerko.features.download_citations_max_count`
- `KERKO_FEEDS` → `kerko.feeds.formats`
- `KERKO_FEEDS_FIELDS` → `kerko.feeds.fields`
- `KERKO_FEEDS_MAX_DAYS` → `kerko.feeds.max_days`
- `KERKO_FEEDS_REJECT_ANY` → `kerko.feeds.reject_any`
- `KERKO_FEEDS_REQUIRE_ANY` → `kerko.feeds.require_any`
- `KERKO_FULLTEXT_SEARCH` → `kerko.search.fulltext`
- `KERKO_HIGHWIREPRESS_TAGS` → `kerko.meta.highwirepress_tags`
- `KERKO_JQUERY_VERSION` → `kerko.assets.jquery_version`
- `KERKO_PAGE_LEN` → `kerko.pagination.page_len`
- `KERKO_PAGER_LINKS` → `kerko.pagination.pager_links`
- `KERKO_POPPER_VERSION` → `kerko.assets.popper_version`
- `KERKO_PRINT_CITATIONS_LINK` → `kerko.features.print_citations_link`
- `KERKO_PRINT_CITATIONS_MAX_COUNT` → `kerko.features.print_citations_max_count`
- `KERKO_PRINT_ITEM_LINK` → `kerko.features.print_item_link`
- `KERKO_RELATIONS_INITIAL_LIMIT` → `kerko.features.relations_initial_limit`
- `KERKO_RELATIONS_LINKS` → `kerko.features.relations_links`
- `KERKO_RELATIONS_SORT` → `kerko.features.relations_sort`
- `KERKO_RESULTS_ABSTRACTS_MAX_LENGTH` → `kerko.features.results_abstracts_max_length`
- `KERKO_RESULTS_ABSTRACTS_MAX_LENGTH_LEEWAY` → `kerko.features.results_abstracts_max_length_leeway`
- `KERKO_RESULTS_ABSTRACTS` → `kerko.features.results_abstracts`
- `KERKO_RESULTS_ABSTRACTS_TOGGLER` → `kerko.features.results_abstracts_toggler`
- `KERKO_RESULTS_ATTACHMENT_LINKS` → `kerko.features.results_attachment_links`
- `KERKO_RESULTS_FIELDS` → `kerko.search.result_fields`
- `KERKO_RESULTS_URL_LINKS` → `kerko.features.results_url_links`
- `KERKO_TEMPLATE_ATOM_FEED` → `kerko.templates.atom_feed`
- `KERKO_TEMPLATE_BASE` → `kerko.templates.base`
- `KERKO_TEMPLATE_ITEM` → `kerko.templates.item`
- `KERKO_TEMPLATE_LAYOUT` → `kerko.templates.layout`
- `KERKO_TEMPLATE_SEARCH` → `kerko.templates.search`
- `KERKO_TEMPLATE_SEARCH_ITEM` → `kerko.templates.search_item`
- `KERKO_TITLE` → `kerko.meta.title`
- `KERKO_WHOOSH_LANGUAGE` → `kerko.search.whoosh_language`
- `KERKO_WITH_JQUERY` → `kerko.assets.with_jquery`
- `KERKO_WITH_POPPER` → `kerko.assets.with_popper`
- `KERKO_ZOTERO_API_KEY` → `ZOTERO_API_KEY`
- `KERKO_ZOTERO_BATCH_SIZE` → `kerko.zotero.batch_size`
- `KERKO_ZOTERO_LIBRARY_ID` → `ZOTERO_LIBRARY_ID`
- `KERKO_ZOTERO_LIBRARY_TYPE` → `ZOTERO_LIBRARY_TYPE`
- `KERKO_ZOTERO_LOCALE` → `kerko.zotero.locale`
- `KERKO_ZOTERO_MAX_ATTEMPTS` → `kerko.zotero.max_attempts`
- `KERKO_ZOTERO_WAIT` → `kerko.zotero.wait`
- The Kerko configuration should now be initialized by a call like
`kerko.config_helpers.config_update(app.config, kerko.DEFAULTS)`.
- `Composer.__init__()` now only takes a configuration object as argument
instead of a bunch of arguments. Thus, the initial values of the `Composer`
instance now depend solely on the configuration.
- Remove the `KERKO_USE_TRANSLATIONS` configuration variable. Kerko now relies
on the application's default Babel domain and translation directories. Custom
applications that wish to load Kerko's translations should now add
`kerko.TRANSLATION_DOMAIN` and `kerko.TRANSLATION_DIRECTORIES` to their Babel
configuration.
- The `__init__()` method of `FacetSpec` and its subclasses now only accept
keyword arguments.
- The `sort_key` argument to `FacetSpec.__init__()` is now `sort_by`.
- `Composer` built-in fields `z_dateAdded` and `z_dateModified` are now named
`date_added` and `date_modified` respectively.

*Warning:* Upgrading from version 0.8.x or earlier will require that you rebuild
your search index. Use the following commands, then restart the application:

bash
flask kerko clean index
flask kerko sync index


Features:

- Add expand/collapse actions on facet values, allowing full exploration of
hierarchical facets without changing the current search.
- Add an optional initial limit on the number of values to show under each
facet. When the initial limit is reached, a "show more" button allows the user
to expand the full list.
- Add Atom syndication feeds.
- Allow searching items by their Zotero key.
- Add XML sitemap.
- Add the `kerko count` CLI command (mostly meant for development purposes).

Bug fixes:

- Fix last sync time not displayed at the bottom of search results when
`KERKO_PRINT_CITATIONS_LINK` and `KERKO_DOWNLOAD_CITATIONS_LINK` are both set
to `False`.
- Fix the `kerko sync` CLI command not returning an error code with some types
of failures.
- Fix invalid HTML in help modal.

Other changes:

- Handle new Zotero fields introduced with the new 'preprint' item type.
- Apply a boost factor to DOI, ISBN and ISSN fields extracted from the Extra
field (previously, only the dedicated Zotero fields had a boost factor).
- Under each facet, always show the facet's active filters first.
- Make facet values with long labels easier to read (by indenting wrapped lines
to the right of the checkbox).
- Add link to full bibliography from item pages.
- Add blocks in templates to facilitate theming.
- Remove `page` parameter from pagination links when `page=1`.
- Improve documentation.
- Make sync and schema-related error messages more helpful and user-friendly.
- Move pydocstyle config to `pyproject.toml`.
- Tag package as compatible with Python 3.10 and 3.11.
- Remove leftover code related to Python versions older than 3.7

Backwards incompatible changes:

- Remove the `KERKO_FACET_COLLAPSING` option. The new initial limit on facets
values made this feature largely redundant.
- Remove the `collapsible` param from the `FacetSpec` class.
- Remove support for configuration variables `KERKO_ZOTERO_START` and
`KERKO_ZOTERO_END` (were only used for development and no longer practical).

Changes that might break custom themes:

- The HTML markup and CSS styles of expand/collapse buttons have changed.
- The parameters of the `facet`, `facet_item`, `facet_items`, `facet_fields`
template macros have changed.
- The `facets-container`, `facets`, and `facets-modal-body` element ids are now
required in `search.html.jinja2`.

Possibly backwards incompatible changes (more or less internal API changes):

- Rewrote the `criteria` module. `Criteria.keywords` and `Criteria.filters` work
pretty much as before, but everything else has changed.
- Rewrote the `query` module, which had organically grown into an tangled mess,
now replaced with the `searcher` module. This new API is completely different.
- Adapted view code to the above-mentioned `searcher` API.
- Split the monolithic `views` module into multiple modules under `views`
(`item_creators`, `item_facets`, `item_relations`, `routes`, `search`), and
moved `breadbox`, `meta` (as `item_meta`), `pager`, and `sorter` under
`views`.