Nlcodec

Latest version: v0.5

Safety actively analyzes 613705 Python packages for vulnerabilities to keep your Python projects secure.

Scan your dependencies

Page 1 of 2

0.5

- Add `byte` scheme

0.4.0

- Add support for `class` scheme -- for multi-class classification field

0.3.2

- Feature: shrink existing vocabulary to given dataset (useful for parent child transfer)

0.3.1

- Fix `nlcodec` CLI bug
- Improve help messages with epilog
- Add `nlcodec-learn` interface for vocabulary learn over PySpark

0.3.0

- add `nlcodec-freqs` CLI to setup.py
- log time and memory usage for `learn` task
- log BPE merge operations once every 2s instead of all operations
- using`__slots__`: ~25% faster, %30 less memory for BPE with 3M word types
- `nlcodec.db.core` with `Db` and `MultipartDb`
- `nlcodec.db.batch` with `Batch` and `BathIterable`
- CLI `nlcodec.learn` for learning BPE using pyspark
- CLI `nlcodec.bitextdb` to build a database from parallel text

0.2.4

- fix issue with `name` as class property (24, 25)

Page 1 of 2

© 2024 Safety CLI Cybersecurity Inc. All Rights Reserved.