piwheels

tokie

Blazingly fast tokenizer — 50x faster, 10x smaller, 100% accurate

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install tokie

Dependencies

• None

Releases

Version	Released	Bookworm Python 3.11	Trixie Python 3.13	Files
0.1.4	2026-07-24
tokie-0.1.4-cp311-cp311-linux_armv6l.whl (3 MB) tokie-0.1.4-cp311-cp311-linux_armv7l.whl (3 MB) tokie-0.1.4-cp313-cp313-linux_armv6l.whl (3 MB) tokie-0.1.4-cp313-cp313-linux_armv7l.whl (3 MB)
0.1.3	2026-07-24
tokie-0.1.3-cp313-cp313-linux_armv6l.whl (3 MB) tokie-0.1.3-cp313-cp313-linux_armv7l.whl (3 MB)
0.1.2	2026-07-24
tokie-0.1.2-cp311-cp311-linux_armv6l.whl (3 MB) tokie-0.1.2-cp311-cp311-linux_armv7l.whl (3 MB) tokie-0.1.2-cp313-cp313-linux_armv6l.whl (3 MB) tokie-0.1.2-cp313-cp313-linux_armv7l.whl (3 MB)
0.1.1	2026-07-23
tokie-0.1.1-cp311-cp311-linux_armv6l.whl (3 MB) tokie-0.1.1-cp311-cp311-linux_armv7l.whl (3 MB) tokie-0.1.1-cp313-cp313-linux_armv6l.whl (3 MB) tokie-0.1.1-cp313-cp313-linux_armv7l.whl (3 MB)
0.1.0	2026-07-23
tokie-0.1.0-cp311-cp311-linux_armv6l.whl (3 MB) tokie-0.1.0-cp311-cp311-linux_armv7l.whl (3 MB) tokie-0.1.0-cp313-cp313-linux_armv6l.whl (3 MB) tokie-0.1.0-cp313-cp313-linux_armv7l.whl (3 MB)
0.0.10	2026-05-28
tokie-0.0.10-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.10-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.10-cp313-cp313-linux_armv6l.whl (2 MB) tokie-0.0.10-cp313-cp313-linux_armv7l.whl (2 MB)
0.0.10a2 pre-release	2026-05-28
tokie-0.0.10a2-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.10a2-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.10a2-cp313-cp313-linux_armv6l.whl (2 MB) tokie-0.0.10a2-cp313-cp313-linux_armv7l.whl (2 MB)
0.0.10a1 pre-release	2026-05-28
0.0.9	2026-05-05
tokie-0.0.9-cp313-cp313-linux_armv6l.whl (2 MB) tokie-0.0.9-cp313-cp313-linux_armv7l.whl (2 MB)
0.0.8	2026-03-30
tokie-0.0.8-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.8-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.8-cp313-cp313-linux_armv7l.whl (2 MB) tokie-0.0.8-cp313-cp313-linux_armv6l.whl (2 MB)
0.0.7	2026-03-26
tokie-0.0.7-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.7-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.7-cp313-cp313-linux_armv7l.whl (2 MB) tokie-0.0.7-cp313-cp313-linux_armv6l.whl (2 MB)
0.0.6	2026-03-26
tokie-0.0.6-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.6-cp311-cp311-linux_armv6l.whl (2 MB)
0.0.5	2026-03-24
tokie-0.0.5-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.5-cp311-cp311-linux_armv7l.whl (2 MB)
0.0.4	2026-03-24
tokie-0.0.4-cp311-cp311-linux_armv6l.whl (2 MB) tokie-0.0.4-cp311-cp311-linux_armv7l.whl (2 MB) tokie-0.0.4-cp313-cp313-linux_armv7l.whl (2 MB) tokie-0.0.4-cp313-cp313-linux_armv6l.whl (2 MB)
0.0.3	2026-03-23

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue