Meguru-tokenizer

Latest version: v0.3.1

Safety actively analyzes 625095 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

0.2.0

see. tutorials

0.1.0

implemented tokenizer
- WhitespaceTokenizer
- SudachiTokenzier
- SentencePieceTokenizer

Links

Releases

Has known vulnerabilities

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.