Japanesetokenizer

Latest version: v1.6

Safety actively analyzes 630130 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 2 of 4

1.3.4

fixed this issue https://github.com/Kensuke-Mitsuzawa/JapaneseTokenizers/issues/36

1.3.3

The bugs in the filtering process, they are fixed.

- issue33 https://github.com/Kensuke-Mitsuzawa/JapaneseTokenizers/issues/33
- issue31 https://github.com/Kensuke-Mitsuzawa/JapaneseTokenizers/issues/31

1.3.2

There is a case which mecab + neologd returns a 10th-additional field. That causes value error inside a process of mecab-wrapper module. Detailed info. in https://github.com/Kensuke-Mitsuzawa/JapaneseTokenizers/pull/28

1.3.1

- Jumanpp interface works faster than the previous version.
- This is only for UNIX distributions. Windows OS is an exception.

1.3.1a1

- Jumanpp interface works faster than the previous version.
- This is only for UNIX distributions. Windows OS is an exception.

1.3.0

- It introduced de-normalization function after tokenization process. (全角英数 -> 半角英数)
- For mecab-config, it detects path to mecab-config automatically
- It fixed a bug of initializing juman-object in python2

Page 2 of 4

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.