piwheels

rftokenizer

A character-wise tokenizer for morphologically rich languages

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install rftokenizer

Dependencies

• xgboost
• scikit-learn
• hyperopt
• numpy
• pandas

Releases

Version	Released	Bookworm Python 3.11	Trixie Python 3.13	Files
3.0.0	2026-06-15
rftokenizer-3.0.0-py3-none-any.whl (38 MB)
2.3.2	2025-03-07
rftokenizer-2.3.2-py3-none-any.whl (75 MB)
2.3.1	2025-03-07
rftokenizer-2.3.1-py3-none-any.whl (77 MB)
2.3.0	2024-12-17
rftokenizer-2.3.0-py3-none-any.whl (75 MB)
2.2.0	2024-02-01
rftokenizer-2.2.0-py3-none-any.whl (66 MB)
2.0.1	2021-08-19
rftokenizer-2.0.1-py3-none-any.whl (59 MB)
2.0.0	2021-08-19
rftokenizer-2.0.0-py3-none-any.whl (59 MB)
1.1.0	2020-05-23
rftokenizer-1.1.0-py3-none-any.whl (45 MB)
1.0.1	2019-08-01
rftokenizer-1.0.1-py3-none-any.whl (57 MB)

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue