Flair

Biomedical Models and Datasets:

Most of the biomedical models and datasets were developed together with the [Knowledge Management in Bioinformatics](https://www.informatik.hu-berlin.de/de/forschung/gebiete/wbi) group at the HU Berlin, in particular leonweber and mariosaenger. [This page](https://github.com/flairNLP/flair/blob/master/resources/docs/HUNFLAIR.md) gives an overview of the new models and datasets, and example tutorials. Some highlights:


Biomedical NER models (1790)

Flair now has pre-trained models for biomedical NER trained over unified versions of 31 different biomedical corpora. Because they are trained on so many different datasets, the models are shown to be very robust with new datasets, outperforming all previously available off-the-shelf datasets. If you want to load a model to detect "diseases" in text for instance, do:

python
make a sentence
sentence = Sentence("Behavioral abnormalities in the Fmr1 KO2 Mouse Model of Fragile X Syndrome")

load disease tagger and predict
tagger = SequenceTagger.load("hunflair-disease")
tagger.predict(sentence)


Done! Let's print the diseases found by the tagger:

python
for entity in sentence.get_spans():
print(entity)

This should print:
~~~
Span [1,2]: "Behavioral abnormalities" [− Labels: Disease (0.6736)]
Span [10,11,12]: "Fragile X Syndrome" [− Labels: Disease (0.99)]
~~~

You can also get one model that finds 5 biomedical entity types (diseases, genes, species, chemicals and cell lines), like this:

python
load bio-NER tagger and predict
tagger = MultiTagger.load("hunflair")
tagger.predict(sentence)

This should print:
~~~
Span [1,2]: "Behavioral abnormalities" [− Labels: Disease (0.6736)]
Span [10,11,12]: "Fragile X Syndrome" [− Labels: Disease (0.99)]
Span [5]: "Fmr1" [− Labels: Gene (0.838)]
Span [7]: "Mouse" [− Labels: Species (0.9979)]
~~~

So it now also finds genes and species. As explained [here](https://github.com/flairNLP/flair/blob/master/resources/docs/HUNFLAIR.md) these models work best if you use them together with a biomedical tokenizer.


Biomedical NER datasets (1790)

Flair now supports 31 biomedical NER datasets out of the box, both in their standard versions as well as the "Huner" splits for reproducibility of experiments. For a full list of datasets, refer to [this page](https://github.com/flairNLP/flair/blob/master/resources/docs/HUNFLAIR_CORPORA.md).

You can load a dataset like this:

python
load one of the bioinformatics corpora
corpus = JNLPBA()

print statistics and one sentence
print(corpus)
print(corpus.train[0])


We also include "huner" corpora that combine many different biomedical datasets into a single corpus. For instance, if you execute the following line:

python
load combined chemicals corpus
corpus = HUNER_CHEMICAL()


This loads a combination of 6 different corpora that contain annotation of chemicals into a single corpus. This allows you to train stronger cross-corpus models since you now combine training data from many sources. See more info [here](https://github.com/flairNLP/flair/blob/master/resources/docs/HUNFLAIR_CORPORA.md#huner-data-sets).


POS model for Portuguese clinical text (1789)

Thanks to LucasFerroHAILab, we now include a model for part-of-speech tagging in Portuguese clinical text. Run this model like this:

python
load your tagger
tagger = SequenceTagger.load('pt-pos-clinical')

example sentence
sentence = Sentence('O vírus Covid causa fortes dores .')
tagger.predict(sentence)
print(sentence)


You can find more details in their paper [here](https://link.springer.com/article/10.1007/s42600-020-00067-7).


Model for negation and speculation in biomedical literature (1758)

Using the BioScope corpus, we trained a model to recognize negation and speculation in biomedical literature. Use it like this:

python
sentence = Sentence("The picture most likely reflects airways disease")

tagger = SequenceTagger.load("negation-speculation")
tagger.predict(sentence)

for entity in sentence.get_spans():
print(entity)


This should print:

~~~
Span [4,5,6,7]: "likely reflects airways disease" [− Labels: SPECULATION (0.9992)]
~~~

Thus indicating that this portion of the sentence is speculation.


Other New Features:

MultiTagger (1791)

We added support for tagging text with multiple models at the same time. This can save memory usage and increase tagging speed.

For instance, if you want to POS tag, chunk, NER and detect frames in your text at the same time, do:

python
load tagger for POS, chunking, NER and frame detection
tagger = MultiTagger.load(['pos', 'upos', 'chunk', 'ner', 'frame'])

example sentence
sentence = Sentence("George Washington was born in Washington")

predict
tagger.predict(sentence)

print(sentence)


This will give you a sentence annotated with 5 different layers of annotation.

Sentence splitting

Flair now includes convenience methods for sentence splitting. For instance, to use segtok to split and tokenize a text into sentences, use the following code:

python
from flair.tokenization import SegtokSentenceSplitter

example text with many sentences
text = "This is a sentence. This is another sentence. I love Berlin."

initialize sentence splitter
splitter = SegtokSentenceSplitter()

use splitter to split text into list of sentences
sentences = splitter.split(text)


We also ship other splitters, such as `SpacySentenceSplitter` (requires SpaCy to be installed).

Japanese tokenization (1786)

Thanks to himkt we now have expanded support for Japanese tokenization in Flair. For instance, use the following code to tokenize a Japanese sentence without installing extra libraries:

python
from flair.data import Sentence
from flair.tokenization import JapaneseTokenizer

init japanese tokenizer
tokenizer = JapaneseTokenizer("janome")

make sentence (and tokenize)
sentence = Sentence("私はベルリンが好き", use_tokenizer=tokenizer)

output tokenized sentence
print(sentence)


One-Cycle Learning (1776)

Thanks to lucaventurini2 Flair one supports one-cycle learning, which may give quicker convergence. For instance, train a model in 20 epochs using the code below:

python
train as always
trainer = ModelTrainer(tagger, corpus)

set one cycle LR as scheduler
trainer.train('onecycle_ner',
scheduler=OneCycleLR,
max_epochs=20)


Improvements:

Changes in convention

Turn on tokenizer by default in `Sentence` object (1806)

The `Sentence` object now executes tokenization (`use_tokenizer=True`) by default:

python
Tokenizes by default
sentence = Sentence("I love Berlin.")
print(sentence)

i.e. this is equivalent to
sentence = Sentence("I love Berlin.", use_tokenizer=True)
print(sentence)

i.e. if you don't want to use tokenization, set it to False
sentence = Sentence("I love Berlin.", use_tokenizer=False)
print(sentence)


`TransformerWordEmbeddings` now handle long documents by default

Previously, so had to set `allow_long_sentences=True` to enable handling of long sequences (greater than 512 subtokens) in `TransformerWordEmbeddings`. This is no longer necessary as this value is now set to `True` by default.


Bug fixes
- Fix serialization of `BytePairEmbeddings` (1802)
- Fix issues with loading models that use `ELMoEmbeddings` (1803)
- Allow longer lengths in transformers that can handle more than 512 subtokens (1804)
- Fix encoding for WASSA datasets (1766)
- Update BPE package (1764)
- Improve documentation (1752 1778)
- Fix evaluation of `TextClassifier` if no `label_type` is passed (1748)
- Remove torch version checks that throw errors (1744)
- Update DaNE dataset URL (1800)
- Fix weight extraction error for empty sentences (1805)

New Features and Enhancements:

TransformerWordEmbeddings can now process long sentences (1680)

Adds a heuristic as a workaround to the max sequence length of some transformer embeddings, making it possible to now embed sequences of arbitrary length if you set `allow_long_sentences=True`, like so:

python
TransformerWordEmbeddings(
allow_long_sentences=True, set allow_long_sentences to True to enable this features
),


Setting random seeds (1671)

It is now possible to set seeds when loading and downsampling corpora, so that the sample is always the same:

python
set a random seed
import random
random.seed(4)

load and downsample corpus
corpus = SENTEVAL_MR(filter_if_longer_than=50).downsample(0.1)

print first sentence of dev and test
print(corpus.dev[0])
print(corpus.test[0])


Make reprojection layer optional (1676)

Makes the reprojection layer optional in SequenceTagger. You can control this behavior through the `reproject_embeddings` parameter. If you set it to `True`, embeddings are reprojected via linear map to identical size. If set to `False`, no reprojection happens. If you set this parameter to an integer, the linear map maps embedding vectors to vectors of this size.

python
tagger with standard reprojection
tagger = SequenceTagger(
hidden_size=256,
[...]
reproject_embeddings=True,
)

tagger without reprojection
tagger = SequenceTagger(
hidden_size=256,
[...]
reproject_embeddings=False,
)

reprojection to vectors of length 128
tagger = SequenceTagger(
hidden_size=256,
[...]
reproject_embeddings=128,
)


Set label name when predicting (1671)

You can now optionally specify the "label name" of the predicted label. This may be useful if you want to for instance run two different NER models on the same sentence:

python
sentence = Sentence('I love Berlin')

load two NER taggers
tagger_1 = SequenceTagger.load('ner')
tagger_2 = SequenceTagger.load('ontonotes-ner')

specify label name of tagger_1 to be 'conll03_ner'
tagger_1.predict(sentence, label_name='conll03_ner')

specify label name of tagger_2 to be 'onto_ner'
tagger_1.predict(sentence, label_name='onto_ner')

print(sentence)


This may be useful if you have multiple ner taggers and wish to tag the same sentence with them. Then you can distinguish between the tags by the taggers. It is also now no longer possible to give the predict method a string - you now must pass a sentence.

Sentence Transformers (1696)

Adds the `SentenceTransformerDocumentEmbeddings` class so you get embeddings from the [`sentence-transformer`](https://github.com/UKPLab/sentence-transformers) library. Use as follows:

python
from flair.data import Sentence
from flair.embeddings import SentenceTransformerDocumentEmbeddings

init embedding
embedding = SentenceTransformerDocumentEmbeddings('bert-base-nli-mean-tokens')

create a sentence
sentence = Sentence('The grass is green .')

embed the sentence
embedding.embed(sentence)


You can find a full list of their pretained models [here](https://docs.google.com/spreadsheets/d/14QplCdTCDwEmTqrn1LH4yrbKvdogK4oQvYO1K1aPR5M/edit#gid=0).

Other enhancements
- Update to transformers 3.0.0 (1727)
- Better Memory mode presets for classification corpora (1701)
- ClassificationDataset now also accepts line with "\t" seperator additionaly to blank spaces (1654)
- Change default fine-tuning in DocumentPoolEmbeddings to "none" (1675)
- Short-circuit the embedding loop (1684)
- Add option to pass kwargs into transformer models when initializing model (1694)


New Datasets and Models

Two new dutch NER models (1687)

The new default model is a BERT-based RNN model with the highest accuracy:

python
from flair.data import Sentence
from flair.models import SequenceTagger

load the default BERT-based model
tagger = SequenceTagger.load('nl-ner')

tag sentence
sentence = Sentence('Ik hou van Amsterdam')
tagger.predict(sentence)


You can also load a Flair-based RNN model (might be faster on some setups):

python
load the default BERT-based model
tagger = SequenceTagger.load('nl-ner-rnn')


Corpus of communicative functions (1683) and pre-trained model (1706)

Adds corpus of communicate functions in scientific literature, described in this [LREC paper](https://www.researchgate.net/publication/339658767_An_Evaluation_Dataset_for_Identifying_Communicative_Functions_of_Sentences_in_English_Scholarly_Papers) and available [here](https://github.com/Alab-NII/FECFevalDataset). Load with:

python
corpus = COMMUNICATIVE_FUNCTIONS()
print(corpus)


We also ship a pre-trained model on this corpus, which you can load with:
python
load communicative function tagger
tagger = TextClassifier.load('communicative-functions')

load communicative function tagger
sentence = Sentence("However, previous approaches are limited in scalability .")

predict and print labels
tagger.predict(sentence)
print(sentence.labels)



Keyword Extraction Corpora (1629) and pre-trained model (1689)

Added 3 datasets available for [keyphrase extraction](https://github.com/midas-research/keyphrase-extraction-as-sequence-labeling-data) via sequence labeling: [Inspec](https://github.com/midas-research/keyphrase-extraction-as-sequence-labeling-data/tree/master/Inspec), [SemEval-2017](https://github.com/midas-research/keyphrase-extraction-as-sequence-labeling-data/tree/master/SemEval-2017) and [Processed SemEval-2010](https://github.com/midas-research/keyphrase-extraction-as-sequence-labeling-data/tree/master/processed_semeval-2010)

Load like this:

python
inspec_corpus = INSPEC()
semeval_2010_corpus = SEMEVAL2010()
semeval_2017 = SEMEVAL2017()


We also ship a pre-trained model on this corpus, which you can load with:

python
load keyphrase tagger
tagger = SequenceTagger.load('keyphrase')

load communicative function tagger
sentence = Sentence("Here, we describe the engineering of a new class of ECHs through the "
"functionalization of non-conductive polymers with a conductive choline-based "
"bio-ionic liquid (Bio-IL).", use_tokenizer=True)

predict and print labels
tagger.predict(sentence)
print(sentence)


Swedish NER (1652)

Add corpus for swedish NER using dataset https://github.com/klintan/swedish-ner-corpus/. Load with:

python
corpus = NER_SWEDISH()
print(corpus)


German Legal Named Entity Recognition (1697)

Adds corpus of legal named entities for German. Load with:
python
corpus = LER_GERMAN()
print(corpus)


Refactoring of evaluation

We made a number of refactorings to the evaluation routines in Flair. In short: whenever possible, we now use the evaluation methods of sklearn (instead of our own implementations which kept getting issues). This applies to text classification and (most) sequence tagging.

A notable exception is "span-F1" which is used to evaluate NER because there is no good way of counting true negatives. After this PR, our implementation should now exactly mirror the original `conlleval` script of the CoNLL-02 challenge. In addition to using our reimplementation, an output file is now automatically generated that can be directly used with the `conlleval` script.

In more detail, this PR makes the following changes:

- `Span` is now a list of `Token` and can now be iterated like a sentence
- `flair.DataLoader` is now used throughout
- The `evaluate()` interface in the `Model` base class is changed so that it no longer requires a data loader, but ran run either over list of `Sentence` or a `Dataset`
- `SequenceTagger.evaluate()` now explicitly distinguishes between F1 and Span-F1. In the latter case, no TN are counted (1663) and a non-sklearn implementation is used.
- In the `evaluate()` method of the `SequenceTagger` and `TextClassifier`, we now explicitly call the `.predict() `method.

Bug fixes:

- Fix figsize issue (1622)
- Allow strings to be passed instead of Path (1637)
- Fix segtok tokenization issue (1653)
- Serialize dropout in `SequenceTagger` (1659)
- Fix serialization error in `DocumentPoolEmbeddings` (1671)
- Fix subtokenization issues in transformers (1674)
- Add new datasets to __init__.py (1677)
- Fix deprecation warnings due to invalid escape sequences. (1678)
- Fix PooledFlairEmbeddings deserialization error (1604)
- Fix transformer tokenizer deserialization (1686)
- Fix issues caused by embedding mode and lambda functions in ELMoEmbeddings (1692)
- Fix serialization error in PooledFlairEmbeddings (1593)
- Fix mean pooling in PooledFlairEmbeddings (1698)
- Fix condition to assign whitespace_after attribute in the build_spacy_tokenizer wraper (1700)
- Fix WIKINER encoding for windows (1713)
- Detect and ignore empty sentences in BERT embeddings (1716)
- Fix error in returning multiple classes (1717)

Transformer Word Embeddings

If you want to embed the words in a sentence with transformers, do it like this:

python
from flair.embeddings import TransformerWordEmbeddings

init embedding
embedding = TransformerWordEmbeddings('bert-base-uncased')

create a sentence
sentence = Sentence('The grass is green .')

embed words in sentence
embedding.embed(sentence)


If instead you want to use RoBERTa, do:

python
from flair.embeddings import TransformerWordEmbeddings

init embedding
embedding = TransformerWordEmbeddings('roberta-base')

create a sentence
sentence = Sentence('The grass is green .')

embed words in sentence
embedding.embed(sentence)


Transformer Document Embeddings

To get a single embedding for the whole document with BERT, do:

python
from flair.embeddings import TransformerDocumentEmbeddings

init embedding
embedding = TransformerDocumentEmbeddings('bert-base-uncased')

create a sentence
sentence = Sentence('The grass is green .')

embed the sentence
embedding.embed(sentence)


If instead you want to use RoBERTa, do:

python
from flair.embeddings import TransformerDocumentEmbeddings

init embedding
embedding = TransformerDocumentEmbeddings('roberta-base')

create a sentence
sentence = Sentence('The grass is green .')

embed the sentence
embedding.embed(sentence)


Text classification by fine-tuning a transformer

Importantly, you can now fine-tune transformers to get state-of-the-art accuracies in text classification tasks.
Use `TransformerDocumentEmbeddings` for this and set `fine_tune=True`. Then, use the following example code:


python
from torch.optim.adam import Adam

from flair.data import Corpus
from flair.datasets import TREC_6
from flair.embeddings import TransformerDocumentEmbeddings
from flair.models import TextClassifier
from flair.trainers import ModelTrainer

1. get the corpus
corpus: Corpus = TREC_6()

2. create the label dictionary
label_dict = corpus.make_label_dictionary()

3. initialize transformer document embeddings (many models are available)
document_embeddings = TransformerDocumentEmbeddings('distilbert-base-uncased', fine_tune=True)

4. create the text classifier
classifier = TextClassifier(document_embeddings, label_dictionary=label_dict)

5. initialize the text classifier trainer with Adam optimizer
trainer = ModelTrainer(classifier, corpus, optimizer=Adam)

6. start the training
trainer.train('resources/taggers/trec',
learning_rate=3e-5, use very small learning rate
mini_batch_size=16,
mini_batch_chunk_size=4, optionally set this if transformer is too much for your machine
max_epochs=5, terminate after 5 epochs
)


New Taggers, Embeddings and Datasets

Flair 0.5 adds a ton of new taggers, embeddings and datasets.

New Taggers

New sentiment models (1613)

We added new sentiment models for English. The new models are trained over a combined corpus of sentiment dataset, including Amazon product reviews. So they should be applicable to more domains than the old sentiment models that were only trained with movie reviews.

There are two new models, a transformer-based model you can load like this:

python
load tagger
classifier = TextClassifier.load('sentiment')

predict for example sentence
sentence = Sentence("enormously entertaining for moviegoers of any age .")
classifier.predict(sentence)

check prediction
print(sentence)


And a faster, slightly less accurate model based on RNNs you can load like this:

python
classifier = TextClassifier.load('sentiment-fast')


Fine-grained POS models for English (1625)

Adds fine-grained POS models for English so you now have the option between 'pos' and 'upos' models for fine-grained and universal dependencies respectively. Load like this:

python
Fine-grained POS model
tagger = SequenceTagger.load('pos')

Fine-grained POS model (fast variant)
tagger = SequenceTagger.load('pos-fast')

Universal POS model
tagger = SequenceTagger.load('upos')

Universal POS model (fast variant)
tagger = SequenceTagger.load('upos-fast')


Added Malayalam POS and XPOS tagger model (1522)

Added taggers for historical German speech and thought (1532)

New Embeddings

Added language models for historical German by redewiedergabe (1507)

Load the language models with:

python
embeddings_forward = FlairEmbeddings('de-historic-rw-forward')
embeddings_backward = FlairEmbeddings('de-historic-rw-backward')


Added Malayalam flair embeddings models (1458)

python
embeddings_forward = FlairEmbeddings('ml-forward')
embeddings_backward = FlairEmbeddings('ml-backward')


Added Flair Embeddings from CLEF HIPE Shared Task (1554)

Adds the recently trained Flair embeddings on historic newspapers for German/English/French provided by the [CLEF HIPE shared task](https://impresso.github.io/CLEF-HIPE-2020/).

New Datasets

Added NER dataset for Finnish (1620)

You can now load a Finnish NER corpus with
python
ner_finnish = flair.datasets.NER_FINNISH()


Added DaNE dataset (1425)

You can now load a Danish NER corpus with
python
dane = flair.datasets.DANE()


Added SentEval classification datasets (1454)

Adds 6 SentEval classification datasets to Flair:

python
senteval_corpus_1 = flair.datasets.SENTEVAL_CR()
senteval_corpus_2 = flair.datasets.SENTEVAL_MR()
senteval_corpus_3 = flair.datasets.SENTEVAL_SUBJ()
senteval_corpus_4 = flair.datasets.SENTEVAL_MPQA()
senteval_corpus_5 = flair.datasets.SENTEVAL_SST_BINARY()
senteval_corpus_6 = flair.datasets.SENTEVAL_SST_GRANULAR()


Added Sentiment Datasets (1545)

Adds two new sentiment datasets to Flair, namely AMAZON_REVIEWS, a very large corpus of Amazon reviews with sentiment labels, and SENTIMENT_140, a corpus of tweets labeled with sentiment.

python
amazon_reviews = flair.datasets.AMAZON_REVIEWS()
sentiment_140 = flair.datasets.SENTIMENT_140()


Added BIOfid dataset (1589)
python
biofid = flair.datasets.BIOFID()


Refactorings

Any DataPoint can now be labeled (1450)

Refactored the `DataPoint` class and classes that inherit from it (`Token`, `Sentence`, `Image`, `Span`, etc.) so that all have the same methods for adding and accessing labels.

- `DataPoint` base class now defined labeling methods (closes 1449)
- Labels can no longer be passed to `Sentence` constructor, so instead of:
python
sentence_1 = Sentence("this is great", labels=[Label("POSITIVE")])

you should now do:
python
sentence_1 = Sentence("this is great")
sentence_1.add_label('sentiment', 'POSITIVE')

or:
python
sentence_1 = Sentence("this is great").add_label('sentiment', 'POSITIVE')


Note that Sentence labels now have a `label_type` (in the example that's 'sentiment').

- The `Corpus` method `_get_class_to_count` is renamed to `_count_sentence_labels`
- The `Corpus` method `_get_tag_to_count` is renamed to `_count_token_labels`
- `Span` is now a `DataPoint` (so it has an `embedding` and `labels`)

Embeddings module was split into smaller submodules (1588)

Split the previously huge `embeddings.py` into several submodules organized in an `embeddings/` folder. The submodules are:

- `token.py` for all `TokenEmbeddings` classes
- `document.py` for all `DocumentEmbeddings` classes
- `image.py` for all `ImageEmbeddings` classes
- `legacy.py` for embeddings that are now deprecated
- `base.py` for remaining basic classes

All embeddings are still exposed through the embeddings package, so the command to load them doesn't change, e.g.:

python
from flair.embeddings import FlairEmbeddings
embeddings = FlairEmbeddings('news-forward')

so specifying the submodule is not needed.

Datasets module was split into smaller submodules (1510)

Split the previously huge `datasets.py` into several submodules organized in a `datasets/` folder. The submodules are:

- `sequence_labeling.py` for all sequence labeling datasets
- `document_classification.py` for all document classification datasets
- `treebanks.py` for all dependency parsed corpora (UD treebanks)
- `text_text.py` for all bi-text datasets (currently only parallel corpora)
- `text_image.py` for all paired text-image datasets (currently only Feidegger)
- `base.py` for remaining basic classes

All datasets are still exposed through the datasets package, so it is still possible to load corpora with
python
from flair.datasets import TREC_6

without specifying the submodule.

Other refactorings

- Refactor datasets for code legibility (1394)

Small refactorings on `flair.datasets` for easier code legibility and fewer redundancies, removing about 100 lines of code: (1) Moved the default sampling logic from all corpora classes to the parent `Corpus` class. You can now instantiate a `Corpus` only with a train file which will trigger the sampling. (2) Moved the default logic for identifying train, dev and test files into a dedicated method to avoid duplicates in code.

- Extend string output of Sentence (1452)

Other

New Features

Add option to specify document delimiter for language model training (1541)

You now have the option of specifying a document_delimiter when training a LanguageModel. Say, you have a corpus of textual lists and use "[SEP]" to mark boundaries between two lists, like this:


Colors:
- blue
- green
- red
[SEP]
Cities:
- Berlin
- Munich
[SEP]
...


Then you can now train a language model by setting the `document_delimiter` in the `TextCorpus` and `LanguageModel` objects. This will make sure only documents as a whole will get shuffled during training (i.e. the lists in the above example):

python
your document delimiter
delimiter = '[SEP]'

set it when you load the corpus
corpus = TextCorpus(
"data/corpora/conala-corpus/",
dictionary,
is_forward_lm,
character_level=True,
document_delimiter=delimiter,
)

set it when you init the language model
language_model = LanguageModel(
dictionary,
is_forward_lm=True,
hidden_size=512,
nlayers=1,
document_delimiter=delimiter
)

train your language model as always
trainer = LanguageModelTrainer(language_model, corpus)

Allow column delimiter to be set in ColumnCorpus (1526)

Added the possibility to set a different column delimite for `ColumnCorpus`, i.e.

python
corpus = ColumnCorpus(
Path("/path/to/corpus/"),
column_format={0: 'text', 1: 'ner'},
column_delimiter='\t', set a different delimiter
)


if you want to read a tab-separated column corpus.

Improvements in classification corpus datasets (1545)

There are a number of improvements for the `ClassificationCorpus` and `ClassificationDataset` classes:
- It is now possible to select from three memory modes ('full', 'partial' and 'disk'). Use full if the entire dataset and all objects fit into memory. Use 'partial' if it doesn't and use 'disk' if even 'partial' does not fit.
- It is also now possible to provide "name maps" to rename labels in datasets. For instance, some sentiment analysis datasets use '0' and '1' as labels, while some others use 'POSITIVE' and 'NEGATIVE'. By providing name maps you can rename labels so they are consistent across datasets.
- You can now choose which splits to downsample (for instance you might want to downsample 'train' and 'dev' but not 'test')
- You can now specify the option "filter_if_longer_than", to filter all sentences that have more than the number of provided whitespaces. This is useful to limit corpus size as some sentiment analysis datasets are gigantic.

Added different ways to combine ELMo layers (1547)

Improved default annealing scheme to anneal against score and loss (1570)

Add new scheduler that uses dev score as main metric to anneal against, but additionally uses dev loss in case two epochs have the same dev score.

Added option for hidden state position in FlairEmbeddings (1571)

Adds the option to choose which hidden state to use in FlairEmbeddings: either the state at the end of each word, or the state at the whitespace after. Default is the state at the whitespace after.

You can change the default like this:
python
embeddings = FlairEmbeddings('news-forward', with_whitespace=False)


This configuration seems to be better for syntactic tasks. For POS tagging, it seems that you should set `with_whitespace=False`. For instance, on UD_ENGLISH POS-tagging, we get **96.56 +- 0.03** with whitespace and **96.72 +- 0.04** without, averaged over three runs.

See the discussion in 1362 for more details.

Other features

- Added the option of passing different tokenizers when loading classification datasets (1579)

- Added option for true whitespaces in ColumnCorpus 1583

- Configurable cache_root from environment variable (507)

Performance improvements

- Improve performance for loading not-in-memory corpus (1413)

- A new lmdb based alternative backend for word embeddings (1515 1536)

- Slim down requirements (1419)

Bug Fixes

- Fix issue where flair was crashing for cpu only version of pytorch (1393 1418)

- Fix GPU memory error in PooledFlairEmbeddings (1417)

- Various small fixes (1402 1533 1511 1560 1616)

- Improve documentation (1446 1447 1520 1525 1556)

- Fix various issues in classification datasets (1499)

This is an enhancement release that slims down Flair for quicker/easier installation and smaller library size. It also makes Flair compatible with torch 1.4.0 and adds enhancements that reduce model size and improve runtime speed for some embeddings. New features include the ability to steer the precision/recall tradeoff during training of models and support for CamemBERT embeddings.


Memory, Runtime and Dependency Improvements

Slim down dependency tree (1296 1299 1335 1336)

We want to keep list of dependencies of Flair generally small to avoid errors like 1245 and keep the library small and quick to setup. So we removed dependencies that were each only used for one particular feature, namely:
- `ipython` and `ipython-genutils`, only used for visualization settings in iPython notebooks
- `tiny_tokenizer`, used for Japanese tokenization (replaced with instructions for how to install for all users who want to use Japanese tokenizers)
- `pymongo`, used for MongoDB datasets (replaced with instructions for how to install for all users who want to use MongoDB datasets)
- `torchvision`, now only loaded when needed

We also relaxed version requirements for easier installation on Google CoLab (1335 1336)

Dramatic speed-up of BERT embeddings (1308)

shoarora optimized the BERTEmbeddings implementation by removing redundant calls. This was shown to lead to dramatic speed improvements.

Reduce size of models that use WordEmbeddings (1315)

timnon added a method to replace word embeddings in trained model with sqlite database to dramatically reduce memory usage. Creates class `WordEmbeedingsStore` which can be used to replace a `WordEmbeddings`-instance in a flair model via duck-typing. By using this, timnon was able to reduce our ner-servers memory consumption from 6gig to 600mb (10x decrease) by adding a few lines of code. It can be tested using the following lines (also in the docstring). First create a headless version of a model without word embeddings:

python
from flair.inference_utils import WordEmbeddingsStore
from flair.models import SequenceTagger
import pickle
tagger = SequenceTagger.load("multi-ner-fast")
WordEmbeddingsStore.create_stores(tagger)
pickle.dump(tagger, open("multi-ner-fast-headless.pickle", "wb"))

and then to run the stored headless model without word embeddings, use:
python
from flair.data import Sentence
tagger = pickle.load(open("multi-ner-fast-headless.pickle", "rb"))
WordEmbeddingsStore.load_stores(tagger)
text = "Schade um den Ameisenbären. Lukas Bärfuss veröffentlicht Erzählungen aus zwanzig Jahren."
sentence = Sentence(text)
tagger.predict(sentence)



New Features

Prioritize precision/recall or specific classes during training (1345)

klasocki added ways to steer the precision/recall tradeoff during training of models, as well as prioritize certain classes. This option was added to the `SequenceTagger` and the `TextClassifier`.

You can steer precision/recall tradeoff by adding the `beta` parameter, which indicates how many more times recall is important than precision. So if you set `beta=0.5`, precision becomes twice as important than recall. If you set `beta=2`, recall becomes twice as important as precision. Do it like this:

python
tagger = SequenceTagger(
hidden_size=256,
embeddings=embeddings,
tag_dictionary=tag_dictionary,
tag_type=tag_type,
beta=0.5)


If you want to prioritize classes, you can pass a `weight_loss` dictionary to the model classes. For instance, to prioritize learning the NEGATIVE class in a sentiment tagger, do:

python
tagger = TextClassifier(
document_embeddings=embeddings,
label_dictionary=tag_dictionary,
loss_weights={'NEGATIVE': 10.})


which will increase the importance of class NEGATIVE by a factor of 10.

CamemBERT Embeddings (1297)
stefan-it added support for the recently proposed French language model: CamemBERT.

Thanks to the awesome 🤗/Transformers library, CamemBERT can be used in Flair like in this example:

python
from flair.data import Sentence
from flair.embeddings import CamembertEmbeddings

embedding = CamembertEmbeddings()

sentence = Sentence("J'aime le camembert !")
embedding.embed(sentence)

for token in sentence.tokens:
print(token.embedding)


Bug fixes and enhancements

- Fix new RNN format for torch 1.4.0 (1360, 1382 )
- Fix memory issue in PooledFlairEmbeddings (1337 1339)
- Correct subtoken mapping function for GPT-2 and RoBERTa (1242)
- Update the transformers library to the latest 2.3 version (1333)
- Add staticmethod decorator to some functions (1257)
- Add a warning if validation data is too small (1115)
- Remove leftover printline from MUSE embeddings (1224)
- Correct generate_text() UTF-8 conversion (1238)
- Clarify documentation (1295 1332)
- Replace sklearn by scikit-learn (1321)
- Fix off-by-one error in progress logging (1334)
- Fix typo and annotation (1341)
- Various improvements (1347)
- Make load_big_file work with read-only file (1353)
- Rename tiny_tokenizer to konoha (1363)
- Make test loss plotting optional (1372)
- Add pretty print function for Dictionary (1375)

Release 0.4.4 introduces dramatic improvements in inference speed for taggers (thanks to many contributions by pommedeterresautee), Flair embeddings in 300 languages (thanks stefan-it), modular tokenization and many new features and refactorings.


Speed optimizations

Many refactorings by pommedeterresautee to improve inference speed of sequence tagger (1038 1053 1068 1093 1130), Flair embeddings (1074 1095 1107 1132 1145), word embeddings (1084),
embeddings memory management (1082 1117), general optimizations (1112) and classification (1187).

The combined improvements **increase inference speed by a factor of 2-3**!

New features

Modular tokenization (1022)

You can now pass custom tokenizers to `Sentence` objects and `Dataset` loaders to use different tokenizers than the included `segtok` library by implementing a tokenizer method. Currently, in-built support exists for whitespace tokenization, segtok tokenization and Japanese tokenization with mecab (requires mecab to be installed). In the future, we expect support for additional external tokenizers to be added.

For instance, if you wish to use Japanese tokanization performed by mecab, you can instantiate the `Sentence` object like this:

python
from flair.data import build_japanese_tokenizer
from flair.data import Sentence

instantiate Japanese tokenizer
japanese_tokenizer = build_japanese_tokenizer()

init sentence and pass this tokenizer
sentence = Sentence("私はベルリンが好きです。", use_tokenizer=japanese_tokenizer)
print(sentence)



Flair Embeddings for 300 languages (1146)

Thanks to stefan-it, there is now a massivey multilingual Flair embeddings model that covers 300 languages. See 1099 for more info on these embeddings and [this repo](https://github.com/stefan-it/flair-lms#multilingual-flair-embeddings) for more details.

This replaces the old multilingual Flair embeddings that were trained for 6 languages. Load them with:

python
embeddings_fw = FlairEmbeddings('multi-forward')
embeddings_bw = FlairEmbeddings('multi-backward')


Multilingual Character Dictionaries (1157)

Adds two multilingual character dictionaries computed by stefan-it.

Load with

python
dictionary = Dictionary.load('chars-large')
print(len(dictionary.idx2item))

dictionary = Dictionary.load('chars-xl')
print(len(dictionary.idx2item))


Batch-growth annealing (1138)

The paper [Don't Decay the Learning Rate, Increase the Batch Size](https://arxiv.org/abs/1711.00489) makes the case for increasing the batch size over time instead of annealing the learning rate.

This version adds the possibility to have arbitrarily large mini-batch sizes with an accumulating gradient strategy. It introduces the parameter `mini_batch_chunk_size` that you can set to break down large mini-batches into smaller chunks for processing purposes.

So let's say you want to have a mini-batch size of 128, but your memory cannot handle more than 32 samples at a time. Then you can train like this:

python
trainer = ModelTrainer(tagger, corpus)
trainer.train(
"path/to/experiment/folder",
set large mini-batch size
mini_batch_size=128,
set chunk size to lower memory requirements
mini_batch_chunk_size=32,
)


Because we now can arbitrarly raise mini-batch size, we can now execute the annealing strategy in the above paper. Do it like this:

python
trainer = ModelTrainer(tagger, corpus)
trainer.train(
"path/to/experiment/folder",
set initial mini-batch size
mini_batch_size=32,
choose batch growth annealing
batch_growth_annealing=True,
)


Document-level sequence labeling (1194)

Introduces the option for reading entire documents into one Sentence object for sequence labeling. This option is now supported for `CONLL_03`, `CONLL_03_GERMAN` and `CONLL_03_DUTCH` datasets which indicate document boundaries.

Here's how to train a model on CoNLL-03 on the document level:

python
read CoNLL-03 with document_as_sequence=True
corpus = CONLL_03(in_memory=True, document_as_sequence=True)

what tag do we want to predict?
tag_type = 'ner'

3. make the tag dictionary from the corpus
tag_dictionary = corpus.make_tag_dictionary(tag_type=tag_type)

init simple tagger with GloVe embeddings
tagger: SequenceTagger = SequenceTagger(
hidden_size=256,
embeddings=WordEmbeddings('glove'),
tag_dictionary=tag_dictionary,
tag_type=tag_type,
)

initialize trainer
from flair.trainers import ModelTrainer

trainer: ModelTrainer = ModelTrainer(tagger, corpus)

start training
trainer.train(
'path/to/your/experiment',
set a much smaller mini-batch size because documents are huge
mini_batch_size=2,
)


Option to evaluate on training split (1202)

Previously, the `ModelTrainer` only allowed monitoring of dev and test splits during training. Now, you can also monitor the train split to better check if your method is overfitting.

Support for Danish tagging (1183)

Adds support for Danish POS and NER thanks to AmaliePauli!

Use like this:

python
from flair.data import Sentence
from flair.models import SequenceTagger

example sentence
sentence = Sentence("København er en fantastisk by .")

load Danish NER model and predict
ner_tagger = SequenceTagger.load('da-ner')
ner_tagger.predict(sentence)

print annotations (NER)
print(sentence.to_tagged_string())

load Danish POS model and predict
pos_tagger = SequenceTagger.load('da-pos')
pos_tagger.predict(sentence)

print annotations (NER + POS)
print(sentence.to_tagged_string())


Support for DistilBERT embeddings (1044)

You can use them like this:

python
from flair.data import Sentence
from flair.embeddings import BertEmbeddings

embeddings = BertEmbeddings("distilbert-base-uncased")

s = Sentence("Berlin and Munich are nice cities .")
embeddings.embed(s)

for token in s.tokens:
print(token.embedding)
print(token.embedding.shape)


MongoDataset for reading text classification data from a Mongo database (1192)

Adds the option of reading data from MongoDB. See [this documentation](https://github.com/zalandoresearch/flair/pull/1192#issuecomment-540015019) on how to use this features.

Feidegger corpus (1199)

Adds a dataset downloader for the Feidegger corpus consisting of text-image pairs. Instantiate the corpus like this:

python
from flair.datasets import FeideggerCorpus

instantiate Feidegger corpus
corpus = FeideggerCorpus()

print a text-image pair
print(corpus.train[0])


Refactorings

Refactor checkpointing mechanism (1101)

Refactored the checkpointing mechanism and slimmed down interfaces / code required to load checkpoints.

In detail:

- The methods `save_checkpoint` and `load_checkpoint` are no longer part of the `flair.nn.Model` interface. Instead, saving and restoring checkpoints is now (fully) performed by the `ModelTrainer`.
- The optimizer state and scheduler state are removed from the `ModelTrainer` constructor since they are no longer required here.
- Loading a checkpoint is now one line of code (previously two lines).

python
1. initialize trainer as always with a model and a corpus
from flair.trainers import ModelTrainer
trainer: ModelTrainer = ModelTrainer(model, corpus)

2. train your model for 2 epochs
trainer.train(
'experiment/folder',
max_epochs=2,
example checkpointing
checkpoint=True,
)

3. load last checkpoint with one line of code
trainer = ModelTrainer.load_checkpoint('experiment/folder/checkpoint.pt', corpus)

4. continue training for 2 extra epochs
trainer.train('experiment/folder_2', max_epochs=4)


Refactor data sampling during training (1154)

Adds a `FlairSampler` interface to better enable passing custom samplers to the `ModelTrainer`.

For instance, if you want to always shuffle your dataset in chunks of 5 to 10 sentences, you provide a sampler like this:

python
your trainer
trainer: ModelTrainer = ModelTrainer(tagger, corpus)

execute training run
trainer.train('path/to/experiment/folder',
max_epochs=150,
sample data in chunks of 5 to 10
sampler=ChunkSampler(block_size=5, plus_window=5)
)


Other refactorings

- Switch everything to batch first mode (1077)

- Refactor classification to be more consistent with SequenceTagger (1151)

- PyTorch-Transformers -> Transformers 1163

- In-place transpose of tensors (1047)


Enhancements

Documentation fixes (1045 1098 1121 1157 1160 1168 )

Add option to set ` rnn_type` used in ` SequenceTagger` (1113)
Accept string as input in NER predict (1142)

Example usage:

python
init tagger
tagger= SequenceTagger.load('ner')

predict over list of strings
sentences = tagger.predict(
[
'George Washington went to Berlin .',
'George Berlin lived in Washington .'
]
)

output predictions
for sentence in sentences:
print(sentence.to_tagged_string())


Enable One-hot Embeddings of other Tags (1191)



Bug fixes

- Fix the learning rate finder (1119)
- Fix OneHotEmbeddings on Cuda (1147)
- Fix encoding error in ` CSVClassificationDataset` (1055)
- Fix encoding errors related to old windows chars (1135)
- Fix length error in ` CharacterEmbeddings` (1088 )
- Fix tokenizer insert empty token to sentence object (1226)
- Ensure `StackedEmbeddings` always has the same embedding order (1114)
- Use $HOME instead of ~ for ` cache_root` (1134)

Release 0.4.3 includes a host of new features including transformer-based embeddings (roBERTa, XLNet, XLM, etc.), fine-tuneable `FlairEmbeddings`, crosslingual MUSE embeddings, new data loading/sampling methods, speed/memory optimizations, bug fixes and enhancements. It also begins a refactoring of interfaces that prepares more general applicability of Flair to other types of downstream tasks.

Embeddings

Transformer embeddings (941 972 993)

Updates the old `pytorch-pretrained-BERT` library to the latest version of `pytorch-transformers` to support various new Transformer-based architectures for embeddings.

A total of 7 (new/updated) transformer-based embeddings can be used in Flair now:

python
from flair.embeddings import (
BertEmbeddings,
OpenAIGPTEmbeddings,
OpenAIGPT2Embeddings,
TransformerXLEmbeddings,
XLNetEmbeddings,
XLMEmbeddings,
RoBERTaEmbeddings,
)

bert_embeddings = BertEmbeddings()
gpt1_embeddings = OpenAIGPTEmbeddings()
gpt2_embeddings = OpenAIGPT2Embeddings()
txl_embeddings = TransformerXLEmbeddings()
xlnet_embeddings = XLNetEmbeddings()
xlm_embeddings = XLMEmbeddings()
roberta_embeddings = RoBERTaEmbeddings()


Detailed benchmarks on the downsampled CoNLL-2003 NER dataset for English can be found in 873 .

Crosslingual MUSE Embeddings (853)

Use the new `MuseCrosslingualEmbeddings` class to embed any sentence in one of 30 languages into the same embedding space. Behind the scenes the class first does language detection of the sentence to be embedded, and then embeds it with the appropriate language embeddings. If you train a classifier or sequence labeler with (only) this class, it will automatically work across all 30 languages, though quality may widely vary.

Here's how to embed:
python
initialize embeddings
embeddings = MuseCrosslingualEmbeddings()

two sentences in different languages
sentence_1 = Sentence("This red shoe is new .")
sentence_2 = Sentence("Dieser rote Schuh ist rot .")

language code is auto-detected
print(sentence_1.get_language_code())
print(sentence_2.get_language_code())

embed sentences
embeddings.embed([sentence_1, sentence_2])

print similarities
cos = torch.nn.CosineSimilarity(dim=0, eps=1e-6)
for token_1, token_2 in zip (sentence_1, sentence_2):
print(f"'{token_1.text}' and '{token_2.text}' similarity: {cos(token_1.embedding, token_2.embedding)}")



FastTextEmbeddings (879 )

Adds `FastTextEmbeddings` capable of handling for oov words. Be warned though that these embeddings are huge. `BytePairEmbeddings` are much smaller and reportedly of similar quality so it is probably advisable to use those instead.

Fine-tuneable FlairEmbeddings (922)

You can now fine-tune FlairEmbeddings on downstream tasks. You can **fine-tune an existing LM** by simply passing the `fine_tune` parameter in the `FlairEmbeddings` constructor, like this:

python
embeddings = FlairEmbeddings('news-foward', fine_tune=True)


You can also use this option to **task-train a wholly new language model** by passing an empty `LanguageModel` to the `FlairEmbeddings` constructor and the `fine_tune` parameter, like this:

python
make an empty language model
language_model = LanguageModel(
Dictionary.load('chars'),
is_forward_lm=True,
hidden_size=256,
nlayers=1)

init FlairEmbeddings to task-train this model
embeddings = FlairEmbeddings(language_model, fine_tune=True)



Optimizations

Automatic mixed precision support (934)

Mixed precision training can significantly speed up training. It can now be enabled by setting `use_amp=True` in the trainer classes. For instance for training language models you can do:

python
train your language model
trainer = LanguageModelTrainer(language_model, corpus)

trainer.train('resources/taggers/language_model',
sequence_length=256,
mini_batch_size=256,
max_epochs=10,
use_amp=True)


In our experiments, we saw 3x speedup of training large language models though results vary depending on your model size and experimental setup.

Control memory / speed tradeoff during training (891 809).

This release introduces the `embeddings_storage_mode` parameter to the `ModelTrainer` class and `predict()` methods. This parameter can be one of 'none', 'cpu' and 'gpu' and allows you to control the tradeoff between memory usage and speed during training:

- If set to '**none**' all embeddings are deleted after usage - this has lowest memory requirements but means that embeddings need to be recomputed at each epoch of training potentially causing a slowdown.
- If set to '**cpu**' all embeddings are moved to CPU memory after usage. During training, this means that they only need to be moved back to GPU for the forward pass, and not recomputed so in many cases this is faster, but requires memory.
- If set to '**gpu**' all embeddings stay on GPU memory after computation. This eliminates memory shuffling during training, causing a speedup. However this option requires enough GPU memory to be available for all embeddings of the dataset.

To use this option during training, simply set the parameter:

python
initialize trainer
trainer: ModelTrainer = ModelTrainer(tagger, corpus)
trainer.train(
"path/to/your/model",
embeddings_storage_mode='gpu',
)


This release also removes the `FlairEmbeddings`-specific disk-caching mechanism. In the future, a more general caching mechanism applicable to all embedding types may potentially be added as a fourth memory management option.

Speed-ups on in-memory datasets (792)

A new `DataLoader` abstract base class used in Flair will speed up data loading for in-memory datasets.


Refactoring of interfaces (891 843)

This release also slims down interfaces of `flair.nn.Model` and adds a new `DataPoint` interface that is currently implemented by the `Token` and `Sentence` classes. The idea is to widen the applicability of Flair to other data types and other tasks. In the future, the `DataPoint` interface will for example also be implemented by an `Image` object and new downstream tasks added to Flair.

The release also slims down the `evaluate()` method in the `flair.nn.Model` interface to take a `DataLoader` instead of a group of parameters. And refactors the logging header logic. Both refactorings prepare adding new new downstream tasks to Flair in the near future.

Other features

Training Classifiers with CSV files (826 952 967)

Adds the `CSVClassificationCorpus` so you can train classifiers directly from CSVs instead of first having to convert to FastText format. To load a CSV, you need to pass a `column_name_map` (like in `ColumnCorpus`), which indicates which column(s) in the CSV holds the text and which field(s) the label(s):

python
corpus = CSVClassificationCorpus(
path to the data folder containing train / test / dev files
data_folder='path/to/data',
indicates which columns are text and labels
column_name_map={4: "text", 1: "label_topic", 2: "label_subtopic"},
if CSV has a header, you can skip it
skip_header=True)


Data sampling (908)

We added the first (of many) data samplers that can be passed to the `ModelTrainer` to influence training. The `ImbalancedClassificationDatasetSampler` for instance will upsample rare classes and downsample common classes in a classification dataset. It may potentially help with imbalanced datasets. Call like this:
python
initialize trainer
trainer: ModelTrainer = ModelTrainer(tagger, corpus)
trainer.train(
'path/to/folder',
learning_rate=0.1,
mini_batch_size=32,
sampler=ImbalancedClassificationDatasetSampler,
)

There are two experimental chunk samplers (`ChunkSampler` and `ExpandingChunkSampler`) split a dataset into chunks and shuffle them. This preserves some ordering of the original data while also randomizing the data.


Visualization

- Adds HTML vizualization of sequence labeling (933). Call like this:
python
from flair.visual.ner_html import render_ner_html

tagger = SequenceTagger.load('ner')

sentence = Sentence(
"Thibaut Pinot's challenge ended on Friday due to injury, and then Julian Alaphilippe saw "
"his lead fall away. The BBC's Hugh Schofield in Paris reflects on 34 years of hurt."
)

tagger.predict(sentence)
html = render_ner_html(sentence)

with open("sentence.html", "w") as writer:
writer.write(html)


- Plotter now returns images for use in iPython notebooks (943)
- Initial TensorBoard support (924)
- Add pointer to Flair Visualizer (1014)

Additional parameterization options

- `CharacterEmbeddings` now let you specify number of hidden states and embedding size (834)
python
embedding = CharacterEmbedding(char_embedding_dim=64, hidden_size_char=64)

- Adds configuration option for minimal learning rate stopping criterion (871)
- `num_workers` is a parameter of `LanguageModelTrainer` (962 )

Bug fixes / enhancements
- Updates old pretrained models to remove old bugs / performance issues (1017)
- Fix error in RNN initialization in `DocumentRNNEmbeddings` (793)
- `ELMoEmbeddings` now use `flair.device` param (825)
- Fix download of TREC_6 dataset (896)
- Fix download of UD_GERMAN-HDT (980)
- Fix download of WikiNER_German (1006)
- Fix error in `ColumnCorpus` in which words that begin with hashtags were skipped as comments (956)
- Fix `max_tokens_per_do`c param in `ClassificationCorpus` (991)
- Simplify split rule in `ColumnCorpus` (990)
- Fix import error message for `ELMoEmbeddings` (1019)
- References to Persian language unified across embeddings (773)
- Updates most pre-trained models fixing quality issues / bugs (800)
- Clarifications in documentation (803 860 868)
- Fixes infinite loop for tokens without startpos (1030)

Enhancements
- Adds a learnable initial hidden state to `SequenceTagger` (899)
- Now keeps order of sentences in mini-batch when embedding (866)
- `SequenceTagger` now optionally returns a distribution of tag probabilities over all classes (782 949 1016)
- The model trainer now outputs a 'test.tsv' file that contains prediction of final model when done training (771 )
- Releases logging handler when finishing training a model (799)
- Fixes `bad_epochs` in training logs and no longer evaluates on test data at each epoch by default (818 )
- Convenience method to remove all empty sentences from a corpus (795)