fast-tokenizer-python

PaddleNLP Fast Tokenizer Library written in C++

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install fast-tokenizer-python

Dependencies

  • None

Releases

Version Released Bookworm
Python 3.11
Trixie
Python 3.13
Files
1.0.2.post1 2023-02-19
1.0.2 2023-02-17
1.0.1 2023-01-12
1.0.0 2022-11-18

Issues with this package?

Page last updated 2026-09-03 11:20:11 UTC