tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install tiktoken

Dependencies

Releases

Version Released Bookworm
Python 3.11
Trixie
Python 3.13
Files
0.14.0 2026-08-17    
0.13.0 2026-05-15    
0.12.0 2025-10-06    
0.11.0 2025-08-08    
0.10.0 2025-08-05    
0.9.0 2025-02-14    
0.8.0 2024-10-03    
0.7.0 2024-05-13    
0.6.0 2024-02-09    
0.5.2 2023-12-03    
0.5.1 2023-09-13    
0.5.0 2023-09-12    
0.4.0 2023-05-07    
0.3.3 2023-03-28    
0.3.2 2023-03-17    
0.3.1 2023-03-13    
0.3.0 2023-03-02    
0.2.0 2023-02-03    
0.1.2 2023-01-03    
0.1.1 2022-12-15    

Issues with this package?

Page last updated 2026-10-05 11:56:48 UTC