piwheels

count-tokens

Count number of tokens in text files using tiktoken tokenizer from OpenAI

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install count-tokens

Dependencies

• tiktoken

Releases

Version	Released	Bookworm Python 3.11	Trixie Python 3.13	Files
0.8.3	2026-06-18
count_tokens-0.8.3-py3-none-any.whl (11 KB)
0.8.2	2026-04-19
count_tokens-0.8.2-py3-none-any.whl (9 KB)
0.8.1	2026-02-03
count_tokens-0.8.1-py3-none-any.whl (9 KB)
0.7.3	2025-12-12
count_tokens-0.7.3-py3-none-any.whl (8 KB)
0.7.2	2025-01-09
count_tokens-0.7.2-py3-none-any.whl (4 KB)
0.7.0	2023-09-26
count_tokens-0.7.0-py3-none-any.whl (4 KB)
0.6.0	2023-09-26
count_tokens-0.6.0-py3-none-any.whl (4 KB)
0.5.0	2023-09-26
count_tokens-0.5.0-py3-none-any.whl (4 KB)
0.4.0	2023-07-01
count_tokens-0.4.0-py3-none-any.whl (3 KB)
0.3.0	2023-06-28
count_tokens-0.3.0-py3-none-any.whl (3 KB)
0.2.0	2023-06-28
count_tokens-0.2.0-py3-none-any.whl (3 KB)
0.1.0	2023-06-28
count_tokens-0.1.0-py3-none-any.whl (3 KB)

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue