Typesense

Latest version: v0.20.0

Safety actively analyzes 624001 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 5

26.0

This release contains important new features, performance improvements and bug fixes.

New Versioning Scheme

Starting with this release, we're dropping the `0.x.y` versioning scheming and switching to a `x.y` versioning scheme.

So we're going from `0.25 --> 26.0`.

Typesense has been production-ready for a few years now, and is actively used at scale in production, serving billions of search requests per month just on Typesense Cloud, and several billions more in self-hosted clusters.

We originally intended the `0.x` versioning scheme to communicate that there might be backward in-compatible changes between versions. In reality though, we've only had to do two backward incompatible changes over the years. However, the usage of the previous `0.x` versioning scheme seemed to mis-communicate Typesense's production-readiness among new users, causing confusion.

Switching from `0.x` to `1.x` also seemed to mis-communicate the progress and feature-set maturity we've built over the years.
So we decided to simply drop the `0.` and switch to whole numbers for major versions, to convey Typesense's progress over the last 8 years.

New Features

- **Built-in Conversational Search (RAG):** You can now seamlessly run a semantic search and then pass the result to an LLM
for summarizing the result as an answer.
- Built-in support for OpenAI and Cloudflare Workers AI hosted models.
- [Docs](https://typesense.org/docs/26.0/api/conversational-search-rag.html)
- **Image Search:** Search through images using text descriptions of their contents, or perform similarity searches, using the CLIP model.
- [Docs](https://typesense.org/docs/26.0/api/image-search.html)
- **Voice Search:** Capture and send query via voice recordings -- Typesense will transcribe (via Whisper model) and provide search results.
- [Docs](https://typesense.org/docs/26.0/api/voice-search-query.html)
- **JOINs:** Connect one or more collections via common reference fields and join them during query time. This
allows you to model SQL-like relationships elegantly.
- [Docs](https://typesense.org/docs/26.0/api/joins.html)
- **Search Personalization using Historical Queries:** The `vector_query` parameter in Vector Search supports a `qs` parameter (stands for plural of the `q` parameter) that accepts a
comma-separated list of historical search queries. We compute the average embedding of these queries and use that as the vector for search.
- [Docs](https://typesense.org/docs/26.0/api/vector-search.html#searching-with-historical-queries)
- **Analytics:**
- Ability to track queries that don't produce hits. [Docs](https://typesense.org/docs/26.0/api/analytics-query-suggestions.html#no-hits-queries)
- Ability to track counts of document-level analytics (eg: clicks, views, etc) to improve search relevance. [Docs](https://typesense.org/docs/26.0/api/analytics-query-suggestions.html#counting-events-for-popularity)
- **Sorting based on Filter Score:** You can now use `_eval` in `sort_by` and assign scores to records that match particular filters, to boost or bury a set of records together.
the filter expression.
- [Docs](https://typesense.org/docs/26.0/api/search.html#sorting-based-on-filter-score)
- **Stemming:** Allows handling common word variations of the same root word. This is helpful for different word-forms of the same root word (eg: plurals / singular).
- Eg: Searching for `walking`, will also return results with `walk`, `walked`, `walks`, etc when stemming is enabled.
- [Docs](https://typesense.org/docs/26.0/api/collections.html#schema-parameters). See the `stem: true` property under the `fields` parameter.
- **Prefix Filtering:** During filtering, you can now query on records that begin with a given prefix string.
- Eg: `company_name: Acm*` will return names that begin with `acm`.
- Previously, only full-word or full-attribute-value matching was possible in `filter_by` and only `q` supported prefix matches.
- **Stop Words:** Specify a list of common words (e.g.`a`, `am`, `the`, `are`, etc.) that should be excluded from the indexing and search process to improve search relevance and performance.
- [Docs](https://typesense.org/docs/26.0/api/stopwords.html)
- **Curate / Override by Tags:** You can tag override rules with tags and then trigger curation by referring to the rule
by the tag name directly at search time.
- [Docs](https://typesense.org/docs/26.0/api/curation.html#add-tags-to-rules)


Enhancements

- Collection schema changes only block writes now, and reads will be serviced as usual. Previously both reads and writes were blocked.
- **Sort facets alphabetically or by the value of another field:** Sort facet values can now be sorted in
alphabetical order for display via `"facet_by": "phone(sort_by: _alpha:asc)"` or on the value of another field
via `"facet_by": "recipe.name(sort_by: recipe.calories:asc)"`
- **Fetching parent of faceted field:** When you facet on a nested field like `color.name` you can now set
`"facet_return_parent": "color.name"`. This will return the parent color object as parent property in the facet response.
- **Improved faceting and filtering performance:** Query planner has been optimized to handle many common patterns better.
- **NOT contains**: Exclude results that contains a specific string during filtering. For example, `"filter_by": "artist:! Jackson"`
will exclude all documents whose `artist` field value contains the word `jackson`.
- **Excluding IDs via filtering:** The `id` field now support the `:!=` operation, so `"filter_by": "id:!=[id1, id2]"`
will exclude documents that have an `id` value of `id1` or `id2`.
- **Configurable HNSW Parameters:** `M`, `efConstruction` and `efSearch` have been made configurable.
- **Disable typos for numerical tokens:** Use `enable_typos_for_numerical_tokens: false` parameter to disable typos on numerical.
- **Customize URL for OpenAI embedding API:** This allows you to use other OpenAI compatible APIs.
- **Pagination for collections, synonyms & overrides listing:** These API end-points now support `limit` and `offset` GET parameters.
- **Store custom metadata with collection schema:** While creating a collection you can send a `metadata` object field,
which is persisted along with collection schema. This is useful for record keeping.
- **Store metadata with override rules:** Store a `metadata` object within an override, so that the search end-point response
will return the pre-defined metadata associated for that rule. This can can be used to display a message on the front-end.
- **Faster numerical range queries:** You can set `range_index: true` in a field's schema for fast range queries
(this will incur additional memory overhead though).
- Prevent the contents of a field from being stored on-disk via the `store: false` field property.
- Expose information about applied typo tolerance or dropped tokens in `text_match_info` response.
- Option to ignore "not found" error when deleting an object that's already deleted.
- Allow a field which is configured as `index: false` + `optional: false`. Previously this was not allowed.
- Exposed swap usage as a metric in `/metrics.json` API.
- The `/health` API returns additional information about memory/disk exhaustion.
- Support overriding wildcard query via `"q": "*"` in rules.
- Build support for Apple M1 / M2 / M3
- Add option to expand prefix search query via the `expand_query` parameter for suggestion aggregation.
- Auto deletion of expired API keys when the `autodelete: true` property is set during key creation.
- Make the size of search cache configurable via the `--cache-num-entries` server flag. Default is `1000`.
- Add flag for logging search query at the start of req cycle.
- Improved on-disk compaction: prunes older records more aggressively, leading to better bounds on data storage.

Bug Fixes

- Fixed multiple synonym substitutions in query yielding no results.
- Fix `typo_tokens_threshold` not considering the number of grouped hits.
- Fixed odd behavior when `_eval` condition in `sort_by` contained a comma.
- Fixed `object` type auto-creating schema for nested fields even for non-indexed fields.
- Fixed open quote present in search query treated as phrase search.
- Fixed facet by range not working with decimal numbers or with numerical labels or labels that contain spaces.
- Fixed extra new line showing up in the import API response.
- Fixed face range end values being exclusive in nature when it should be inclusive.
- Fixed edge cases in handling unicode in German / Thai locales.
- Fixed facet counts being incorrect when combined with grouping and pinning.
- Fixed highlighting quirks on long documents.
- Fixed inheritance of sort field property for nested fields.
- Fixed propagation of dynamic field properties for child nested fields.
- Fixed some edge cases in phrase search.
- Fixed update doc API returning 200 status code, instead of 201.

Deprecations / behavior changes

There are no depreciation or behavior changes in this version.

Upgrading

Before upgrading your existing Typesense cluster to v26.0, please review the behavior changes above to prepare your application for the upgrade.

We'd recommend testing on your development / staging environments before upgrading.

Typesense Cloud

If you're on Typesense Cloud:

1. Go to [https://cloud.typesense.org/clusters](https://cloud.typesense.org/clusters).
2. Click on your cluster
3. Click on "Cluster Configuration" on the left-side pane, and then click on "Modify"
4. Select a new Typesense Server version in the dropdown
5. Schedule a time for the upgrade.

Self Hosted

If you're self-hosting Typesense, here are instructions on how to upgrade: https://typesense.org/docs/26.0/api/#self-hosted

Downgrading

Once you upgrade to `v26` of Typesense Server, you can only downgrade back to `v0.25.x`.

Documentation

View the complete API documentation for this release here: [https://typesense.org/docs/26.0/api/](https://typesense.org/docs/26.0/api/)

0.25.2

This release fixes some important bugs identified in `v0.25.1` and also adds several enhancements to existing features. So we highly recommend upgrading to it.

Enhancements

- New search parameter flag `prioritize_num_matching_fields` that allows you to configure whether the text match score should consider number of matching fields as a ranking criteria. This defaults to `true`.
- Unload embedding model and free memory when no collections use a given model.
- Allow the hybrid search / keyword search weight (`alpha`) to be configurable in `vector_query`.
- Allow the direction of drop tokens to be configurable via `drop_tokens_mode` search parameter.
- Handle zero width non-joiner character for Persian.
- Ability to apply vector search only on the results of keyword search.
- Prevent Typesense from loading an embedding model that exceeds available memory.
- Improvements to range faceting: float value support, min/max ranges by leaving range param blank, support spaces in range label.
- Support `remote_embedding_timeout_ms` and `remote_embedding_num_tries` for indexing.
- Better isolation of operations across collections.
- Parameterize compaction of store during collection drop via the `compact_store` parameter.
- Server argument for configuring the periodic DB compaction interval via `db-compaction-interval`. Default: `604800` (seconds).
- When using `group_by`, you can now control if documents with a `null` value in the grouped field should be placed in one group together, or should not be grouped using the `group_missing_values` parameter.

Bug fixes

- Improve precision of HNSW index under frequent deletion and updates of vectors.
- Fixed a regression in analytics aggregation for aggregation intervals configured to > 60 seconds.
- Fixed a bug that prevented two embedding field names from sharing the same prefix.
- Fixed reindexing of old documents with embeddings on collection schema update.
- Fixed group by search that used facet range query.
- Fixed edge cases in indexing and querying of optional nested fields.
- Fixed hanging when PaLM API is down.
- Fixed an edge case in deletion of values from infix search index.
- Fixed hybrid search not considering `hidden_hits` search parameter.
- Tweaked text embedding mean-pooling code to match the vectors generated by Python libraries.
- Fixed truncation of certain float values in facet response.

Upgrade

Before upgrading your existing Typesense cluster to `v0.25.2`, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.25.2 documentation](https://typesense.org/docs/0.25.2/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Cluster Configuration" on the right pane, then "Modify", pick `0.25.2` and schedule a time for your upgrade.

Downgrade

Once you upgrade to `v0.25.x` of Typesense Server the internal structure of the data directory becomes incompatible with older versions of Typesense (`v0.24`).

However, if you need to downgrade to `v0.24`, we've released a special version `v0.24.2` that backports these data structure changes back to `v0.24` while keeping other `v0.24.1` features as is.

So `v0.25.x` can only be downgraded to `v0.24.2`.

But `v0.25.2` can be downgraded to `v0.25.1` and `v0.25.0` if needed.

Documentation

View the complete API documentation for this release here: [https://typesense.org/docs/0.25.2/api/](https://typesense.org/docs/0.25.2/api/)

0.25.1

This release fixes some important bugs identified in the recently released `v0.25.0`. So we highly recommend upgrading to it.

Bug Fixes

- Handle mismatched vector dimensions during indexing
- Allow remote embedders to use multiple per-collection credentials
- Fixed altering of schema involving embedding fields
- Fixed charset in content-type header of export API response
- Fixed an issue in analytics query aggregation that caused crash on empty collections
- Fixed group by on vector search
- Improved error handling of remote embedding APIs
- Fixed tokenizer of embedding models that use XLM-RoBERTa
- Fixed upsert of unchanged docs containing embedding fields, that would cause the embedding field value to be removed
- Fixed text match score calculation to consider the presence of dropped tokens when ranking

Upgrade

Before upgrading your existing Typesense cluster to `v0.25.1`, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.25.1 documentation](https://typesense.org/docs/0.25.1/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Cluster Configuration" on the right pane, then "Modify", pick `0.25.1` and schedule a time for your upgrade.

Downgrade

Once you upgrade to `v0.25` of Typesense Server the internal structure of the data directory becomes incompatible with older versions of Typesense (`v0.24`).

However, if you need to downgrade to `v0.24`, we've released a special version `v0.24.2` that backports these data structure changes back to `v0.24` while keeping other `v0.24.1` features as is.

So `v0.25` can only be downgraded to `v0.24.2`.

But `v0.25.1` can be downgraded to `v0.25.0`.

Documentation

View the complete API documentation for this release here: [https://typesense.org/docs/0.25.1/api/](https://typesense.org/docs/0.25.1/api/)

0.25.0

This release contains exciting new features and important bug fixes.

New Features

- **Semantic Search:** Search for conceptually related terms in your dataset, even if the exact keyword does not exist in your dataset.
- [Demo](https://hn-comments-search.typesense.org) | [Docs](https://typesense.org/docs/0.25.0/api/vector-search.html#semantic-search)
- **Hybrid search:** Combine both keyword and semantic / vector search results in a single query using rank fusion
- [Demo](https://hn-comments-search.typesense.org) | [Docs](https://typesense.org/docs/0.25.0/api/vector-search.html#hybrid-search)
- **Automatic embedding generation:** specify one or more string fields that should be used for generating embeddings during indexing & during search using
state-of-the-art embedding models, optionally [using a GPU](https://typesense.org/docs/0.25.0/api/vector-search.html#using-a-gpu-optional).
- [Example](https://github.com/typesense/showcase-hn-comments-semantic-search/blob/0a10f2ef34e01e79049e7ba42ae8660e80cf524f/scripts/indexDataInTypesense.js#L32-L45) | [Docs](https://typesense.org/docs/0.25.0/api/vector-search.html#using-built-in-models)
- **Integration with OpenAI API, PaLM API and Vertex AI API:** Have Typesense automatically make API calls to remote embedding services like OpenAI / Google, to generate vectors for the JSON data you index in Typesense.
- [Example](https://github.com/typesense/showcase-hn-comments-semantic-search/blob/0a10f2ef34e01e79049e7ba42ae8660e80cf524f/scripts/indexDataInTypesense.js#L49-L67) | [Docs](https://typesense.org/docs/0.25.0/api/vector-search.html#using-openai-api)
- **Query Analytics:** Typesense now supports aggregation of popular search queries which can then be used as insights into query patterns. [Docs](https://typesense.org/docs/0.25.0/api/analytics-query-suggestions.html)
- **Query Suggestions:** You can use historical search terms collected by the Query Analytics feature, to power Query Suggestions.
- [Docs](https://typesense.org/docs/0.25.0/api/analytics-query-suggestions.html#query-suggestions)
- **Update Documents by Query:** You can now update all documents that match a `filter_by` condition
- [Docs](https://typesense.org/docs/0.25.0/api/documents.html#update-by-query)
- **Range faceting:** numerical values can be dynamically faceted at query-time by bucketing them into ranges.
- [Docs](https://typesense.org/docs/0.25.0/api/search.html#faceting-parameters)
- **Pagination using `offset` and `limit`**: This is in addition to the existing `page` and `per_page` mechanism. This new pagination method offers more flexibility and is also useful for GraphQL compatibility.
- [Docs](https://typesense.org/docs/0.25.0/api/search.html#pagination-parameters)

Enhancements

- Resolve field names using wildcard: fields can now be resolved in `facet_by`, `query_by`, `include_fields`, `exclude_fields`,
`highlight_fields` and `highlight_full_fields` when a wildcard expression is used, e.g. `title_*` will match `title_en`.
- Ability to sort grouped hits based on the size of each group, using `sort_by: _group_count:desc`.
- A count is returned for total number of records under each group even if the hits are truncating via `group_limit`.
- The `!=` filtering operation can now be performed against numerical fields. Previously only string fields were supported for this operator.
- Support use of `preset` parameter in embedded API key.
- Support nested dynamic fields.
- Migrated build system to Bazel.
- New server configuration option (`--reset-peers-on-error`) that makes the cluster forcefully refresh its peers when an
unrecoverable clustering error happens due to sudden change of peer IPs. There's also an equivalent
`/operations/reset_peers` API. Be careful while using this option, as it can lead to transient loss of data.

Bug Fixes

- Fixed updates of nested object field values.
- Fix geopoint indexing in nested fields.
- Fixed some special characters not getting highlighted properly in prefix searches.
- Fixed a bug in phrase matches on array.
- Fixed a socket leak on followers of a cluster when import data fails validation.
- Fixed high memory usage incurred in export/import of large datasets.
- Fixed bad unicode characters in export.
- Fixed errors that were caused by presence of bad Japanese unicode characters in import.
- Fixed broken http/2 support on CURL v8.
- Fixed non-curated members of a group appearing in curated override results.
- Fixed override query rule being case-sensitive.
- Fixed phrase search not considering field weights.

Deprecations / behavior changes

There are no depreciation or behavior changes in this version.

Upgrade

Before upgrading your existing Typesense cluster to `v0.25.0`, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.25.0 documentation](https://typesense.org/docs/0.25.0/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Cluster Configuration" on the right pane, then "Modify", pick `0.25.0` and schedule a time for your upgrade.

Downgrade

Once you upgrade to `v0.25` of Typesense Server the internal structure of the data directory becomes incompatible with older versions of Typesense.

However, if you need to downgrade to `v0.24`, we've released a special version `v0.24.2` that backports these data structure changes back to `0.24` while keeping other `0.24.1` features as is.

So `v0.25` can only be downgraded to `v0.24.2`.

Documentation

View the complete API documentation for this release here: [https://typesense.org/docs/0.25.0/api/](https://typesense.org/docs/0.25.0/api/)

0.24.1

This release fixes some important bugs identified in the recently released `v0.24.0`. So we recommend updating to it.

Bug Fixes

- Improved memory footprint of export API.
- Fixed an edge case in deletion of deeply nested fields.
- Fixed weighting of fields in phrase search.
- Handle `NaN` values for geo filter query.
- Fixed indexing of nested geopoint array values.
- Added validation for vector dimensions during indexing.
- Fixed highlighting when both flat/nested form of field is present.
- Fixed an edge case with phrase match on array.
- Fixed an edge case in fuzzy search typo correction.
- Allow null values for optional nested fields.
- Enabling exhaustive search should automatically drop tokens.
- Don't remove empty array of object elements for highlight.
- Restrict the `id` from being a default sorting field.

Download

Please download the appropriate binary archive for your operating system and architecture here: https://typesense.org/downloads/

**NOTE:** The new version is also available on [Typesense Cloud](https://cloud.typesense.org/). If you're already using Typesense Cloud, please reach out to us to have your cluster upgraded to `v0.24.1`.

Upgrade

Before upgrading your existing Typesense cluster to `v0.24.1`, please review the changes above to prepare your application for the upgrade.

- ⚠️ For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.23 documentation](https://typesense.org/docs/0.24.1/api/#upgrading).
- For Typesense Cloud, email us at `contacttypesense.org` and we can do an in-place upgrade for you.

0.24.0

This release contains new features, performance improvements and important bug fixes.

New Features

- **Indexing of nested object fields**: feature must be enabled via the `enable_nested_fields`
option during collection creation.
- **Cross-field OR and complex filter expressions**: a filter query like
`is_premium: true || (price:> 100 && category: shoes)` is now possible.
- **Vector search**: support for both exact & HNSW-based approximate vector searching.
- **Multi-lingual support**: Chinese, Japanese, Korean, Cyrillic and Thai are now supported via the use of field-level
`locale` flag in the collection schema.
- **Optional filtering**: `sort_by` clause can now accept an expression whose result is used for sorting, e.g.
`sort_by=_eval(brand:nike):desc,_text_match:desc`.
- **Preset search configurations**: manage your search query parameters by storing them within Typesense as a "preset",
that you can refer to during query time. This helps in keeping the query parameters hidden away from public view.

Enhancements

- New `text_match_type` parameter that allows you to customize how multi-field text relevancy score is computed.
- Improve performance of large collection deletions.
- Ability to clone a collection schema (without documents), overrides and synonyms.
- New highlight structure that mimics the original document structure. Nested fields are highlighted only in this new
structure, which is returned in a key named `highlight` in the JSON response.
- Allow override rules to be processed past the first match via the `stop_processing` flag (default is `true`).
- Support locale and symbols in synonyms via the `locale` and `symbols_to_index` options during synonym creation.
- Allow cloning of collection schema & linked assets (like synonyms, overrides) from a reference collection.
- Support query replacement action in an override via the `replace_query` option.
- Support an override to be active with a given time window via the `effective_from_ts` and `effective_to_ts` options.
- Support `filter_by` rule in overrides.
- New `--skip-writes` flag for starting Typesense in a mode that does not read writes from the Raft log. This is
useful when the server has crashed due to some recent bad writes that you want to skip over temporarily.
- New `--memory-used-max-percentage` and `--disk-used-max-percentage` flags for preventing writes when a specified
memory/disk threshold is breached.
- Allow the imported documents and their `id`s to be returned in the import response via
the `return_doc` and `return_id` options.
- API for compacting the on-disk database via the `POST /operations/db/compact` end-point.

Bug Fixes

- Fixed some edge cases with schema alteration.
- Fixed a race condition in parallel collection creation that manifested on localhost, especially on Mac.
- Fixed an edge case in highlighting involving non-ASCII unicode characters.
- Fixed sort by string not accounting for accented characters.
- Fixed an edge case in numerical facet field values not being fully removed on deletion.
- Fixed HTTPS POST body buffering removing genuine trailing white space in the chunked payload in some cases.
- Fixed float field validation to handle scientific notation.
- Fixed a bug involving exact filter match on array.
- Fixed a few edge cases that showed up in super-large documents that contained greater than 64,000 tokens.
- Implemented automatic log rotation for RocksDB info logs. Previously, one had to restart the server for truncation or
perform external truncation via `logrotate`.

Deprecations / behavior changes

- The default value for `search_cutoff_ms` is now `30000`, i.e. 30 seconds. If you wish to allow searches to run
longer than that, you can pass a higher value as a search parameter.
- Disable text match score bucketing, if there are more buckets than the number of results found. Previously, results
were being aggregated into a single score.
- The older `highlights` key in the response is deprecated (but is still returned for now). Use the new `highlight`
object in the response for highlighting information.
- The `text_match` key in the response is deprecated (but is still returned for now) in favor of the
new `text_match_info` object that returns fine grain matching information, including the score.

Upgrade

Before upgrading your existing Typesense cluster to `v0.24.0`, please review the behavior changes above to prepare your application for the upgrade.

- For self-hosted Typesense deployment, please refer to the important upgrade section of the [0.24.0 documentation](https://typesense.org/docs/0.24.0/api/#upgrading). This particular version requires a specific set of upgrade steps especially for multi-node clusters.
- For Typesense Cloud, visit your [clusters page](https://cloud.typesense.org/clusters), click on the cluster you want to upgrade, click on "Modify Configuration" on the right pane and schedule a time for your upgrade. Alternatively, you can also email `supporttypesense.org`.

Page 1 of 5

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.