- Tokenization rules can be added to a compiled model
1.1
1.1.0
Added
- Implicit instantiation of core classes - Classes and function for ad hoc creation of tokenization config - Methods to save (pickle) and load (unpickle) compiled Normalizer instance - Wheel for Python 3.9
1.0.6
Changed
- Fixed bug with replacing substring that is not a token
1.0.5
Changed
- Normalizer.data is now exposed as a property - Updated documentation, added performance benchmarks - Installable package is either pure Python or wheel with precompiled Cython
1.0.4
Added
- Normalizer.result['r_map'] attribute - Scripts to build wheels