youtokentome

Unsupervised text tokenizer focused on computational efficiency

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install youtokentome

Dependencies

Releases

Version Released Bullseye
Python 3.9
Bookworm
Python 3.11
Trixie
Python 3.13
Files
1.0.6 2020-02-12      
1.0.5 2019-11-20      
1.0.4 2019-11-19      
1.0.3 2019-11-05      
1.0.3rc1 pre-release 2019-11-05      
1.0.2 2019-10-22      
1.0.1 2019-07-19      
1.0.0 2019-07-15      

Issues with this package?

Page last updated 2026-05-12 16:02:51 UTC