piwheels

machinemind-bpe

A Byte Pair Encoding (BPE) tokenizer package for CPU and GPU.

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install machinemind-bpe

Dependencies

• numpy
• torch
• tqdm
• regex

Releases

Version	Released	Bookworm Python 3.11	Trixie Python 3.13	Files
1.0.1	2024-08-05
machinemind_bpe-1.0.1-py3-none-any.whl (7 KB)
1.0.0	2024-08-05
machinemind_bpe-1.0.0-py3-none-any.whl (7 KB)
0.4.0	2024-08-05
machinemind_bpe-0.4.0-py3-none-any.whl (7 KB)
0.3.0	2024-08-05
machinemind_bpe-0.3.0-py3-none-any.whl (7 KB)

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue