- [9045](https://github.com/rasahq/rasa/issues/9045): The option `model_confidence=linear_norm` is deprecated and will be removed in Rasa Open Source `3.0.0`.
Rasa Open Source `2.3.0` introduced `linear_norm` as a possible value for `model_confidence`
parameter in machine learning components such as `DIETClassifier`, `ResponseSelector` and `TEDPolicy`.
Based on user feedback, we have identified multiple problems with this option.
Therefore, `model_confidence=linear_norm` is now deprecated and
will be removed in Rasa Open Source `3.0.0`. If you were using `model_confidence=linear_norm` for any of the mentioned components,
we recommend to revert it back to `model_confidence=softmax` and re-train the assistant. After re-training,
we also recommend to [re-tune the thresholds for fallback components](./fallback-handoff.mdxfallbacks).
- [9091](https://github.com/rasahq/rasa/issues/9091): The fallback mechanism for spaCy models has now been removed in Rasa `3.0.0`.
Rasa Open Source `2.5.0` introduced support for spaCy 3.0. This introduced a
breaking feature because models would no longer be manually linked. To make
the transition smooth Rasa would rely on the `language` parameter in the
`config.yml` to fallback to a medium spaCy model if no model was configured
for the `SpacyNLP` component. In Rasa Open Source `3.0.0` and onwards the
`SpacyNLP` component will require the model name (like `"en_core_web_md"`)
to be passed explicitly.
Features
- [8724](https://github.com/rasahq/rasa/issues/8724): Added `sasl_mechanism` as an optional configurable parameters for the [Kafka Producer](event-brokers.mdx#kafka-event-broker).
- [8913](https://github.com/rasahq/rasa/issues/8913): Introduces a new policy called [`UnexpecTEDIntentPolicy`](./policies.mdx#unexpected-intent-policy).
`UnexpecTEDIntentPolicy` helps you [review conversations](./conversation-driven-development.mdxreview)
and also allows your bot to react to unexpected user turns in conversations.
It is an auxiliary policy that should only be used in conjunction with
at least one other policy, as the only action that it can trigger
is the special and newly introduced
[`action_unlikely_intent`](./default-actions.mdxaction_unlikely_intent) action.
The auto-configuration will include `UnexpecTEDIntentPolicy` in your
configuration automatically, but you can also include it yourself
in the `policies` section of the configuration:
policies:
- name: UnexpecTEDIntentPolicy
epochs: 200
max_history: 5
As part of the feature, it also introduces:
- [`IntentMaxHistoryTrackerFeaturizer`](./policies.mdx3-intent-max-history)
to featurize the trackers for `UnexpecTEDIntentPolicy`.
- `MultiLabelDotProductLoss` to support `UnexpecTEDIntentPolicy`'s multi-label training objective.
- A new default action called [`action_unlikely_intent`](./default-actions.mdxaction_unlikely_intent).
`rasa test` command has also been adapted to `UnexpecTEDIntentPolicy`:
- If a test story contains `action_unlikely_intent` and the policy ensemble does not trigger it, this leads to
a test error (wrongly predicted action) and the corresponding story will be logged in `failed_test_stories.yml`.
- If the story does not contain `action_unlikely_intent` and Rasa Open Source does predict it then
the prediction of `action_unlikely_intent` will be ignored for the evaluation (and hence not lead
to a prediction error) but the story will be logged in a file called `stories_with_warnings.yml`.
The `rasa data validate` command will warn if `action_unlikely_intent` is
included in the training stories. Accordingly, `YAMLStoryWriter` and `MarkdownStoryWriter` have been updated to not dump `action_unlikely_intent` when writing stories to a file.
:::caution
The introduction of a new default action **breaks backward compatibility of previously trained models**.
It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train
your assistant before trying to use this version.
:::
Improvements
- [8127](https://github.com/rasahq/rasa/issues/8127): Added detailed json schema validation for `UserUttered`, `SlotSet`, `ActionExecuted` and `EntitiesAdded` events both sent and received from the action server, as well as covered at high-level the validation of the rest of the 20 events.
In case the events are invalid, a `ValidationError` will be raised.
- [8232](https://github.com/rasahq/rasa/issues/8232): Users don't need to specify an additional buffer size for sparse featurizers anymore during incremental training.
Space for new sparse features are created dynamically inside the downstream machine learning
models - `DIETClassifier`, `ResponseSelector`. In other words, no extra buffer is created in
advance for additional vocabulary items and space will be dynamically allocated for them inside the model.
This means there's no need to specify `additional_vocabulary_size` for [`CountVectorsFeaturizer`](./components.mdxcountvectorsfeaturizer) or
`number_additional_patterns` for [`RegexFeaturizer`](./components.mdxregexfeaturizer). These parameters are now deprecated.
**Before**
yaml
pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
number_additional_patterns: 100
- name: "CountVectorsFeaturizer"
additional_vocabulary_size: {text: 100, response: 20}
**Now**
yaml
pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CountVectorsFeaturizer"
Also, all custom layers specifically built for machine learning models - `RasaSequenceLayer`, `RasaFeatureCombiningLayer`
and `ConcatenateSparseDenseFeatures` now inherit from `RasaCustomLayer` so that they support flexible incremental training out of the box.
- [8295](https://github.com/rasahq/rasa/issues/8295): Speed up the contradiction check of the [`RulePolicy`](policies.mdx#rule-policy)
by a factor of 3.
- [8801](https://github.com/rasahq/rasa/issues/8801): Change the confidence score assigned by [`FallbackClassifier`](components.mdx#fallbackclassifier) to fallback intent to be the same as the fallback threshold.
- [8926](https://github.com/rasahq/rasa/issues/8926): Issue a UserWarning if a specified **domain folder** contains files that look like YML files but cannot be parsed successfully.
Only invoked if user specifies a folder path in `--domain` paramater. Previously those invalid files in the specified folder were silently ignored.
**Does not apply** to individually specified domain YAML files, e.g. `--domain /some/path/domain.yml`, those being invalid will still raise an exception.
Bugfixes
- [8711](https://github.com/rasahq/rasa/issues/8711): Fix for unnecessary retrain and duplication of folders in the model
Miscellaneous internal changes
- [8241](https://github.com/rasahq/rasa/issues/8241), [#8525](https://github.com/rasahq/rasa/issues/8525), [#8694](https://github.com/rasahq/rasa/issues/8694), [#8704](https://github.com/rasahq/rasa/issues/8704)