piwheels

tokenizer

A tokenizer for Icelandic text

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install tokenizer

Dependencies

• None

Releases

Version	Released	Bullseye Python 3.9	Bookworm Python 3.11	Files
3.5.1	2025-08-28
tokenizer-3.5.1-py3-none-any.whl (80 KB)
3.5.0	2025-08-27
tokenizer-3.5.0-py3-none-any.whl (79 KB)
3.4.5	2024-08-23
tokenizer-3.4.5-py3-none-any.whl (110 KB)
3.4.4	2024-08-07
tokenizer-3.4.4-py3-none-any.whl (111 KB)
3.4.3	2023-08-11
tokenizer-3.4.3-py2.py3-none-any.whl (110 KB)
3.4.2	2022-10-07
tokenizer-3.4.2-py2.py3-none-any.whl (77 KB)
3.4.1	2022-05-03
tokenizer-3.4.1-py2.py3-none-any.whl (90 KB)
3.4.0	2022-03-10
tokenizer-3.4.0-py2.py3-none-any.whl (89 KB)
3.3.2	2021-09-27
tokenizer-3.3.2-py2.py3-none-any.whl (88 KB)
3.3.1	2021-09-08
tokenizer-3.3.1-py2.py3-none-any.whl (88 KB)
3.3.0	2021-09-08
tokenizer-3.3.0-py2.py3-none-any.whl (88 KB)
3.2.0	2021-08-16
tokenizer-3.2.0-py2.py3-none-any.whl (88 KB)
3.1.2	2021-06-02
tokenizer-3.1.2-py2.py3-none-any.whl (76 KB)
3.1.1	2021-05-10
tokenizer-3.1.1-py2.py3-none-any.whl (76 KB)
3.1.0	2021-04-14
tokenizer-3.1.0-py2.py3-none-any.whl (75 KB)
3.0.0	2021-04-09
tokenizer-3.0.0-py2.py3-none-any.whl (75 KB)
2.5.0	2021-03-08
tokenizer-2.5.0-py2.py3-none-any.whl (105 KB)
2.4.0	2020-10-08
tokenizer-2.4.0-py2.py3-none-any.whl (103 KB)
2.3.1	2020-09-21
tokenizer-2.3.1-py2.py3-none-any.whl (114 KB)
2.3.0	2020-09-03
tokenizer-2.3.0-py2.py3-none-any.whl (114 KB)
2.2.0	2020-08-20
tokenizer-2.2.0-py2.py3-none-any.whl (466 KB)
2.1.0	2020-07-02
tokenizer-2.1.0-py2.py3-none-any.whl (107 KB)
2.0.7	2020-06-24
tokenizer-2.0.7-py2.py3-none-any.whl (460 KB)
2.0.6	2020-05-29
tokenizer-2.0.6-py2.py3-none-any.whl (463 KB)
2.0.5	2020-03-26
tokenizer-2.0.5-py2.py3-none-any.whl (103 KB)
2.0.4	2020-02-17
tokenizer-2.0.4-py2.py3-none-any.whl (102 KB)
2.0.3	2019-12-17
tokenizer-2.0.3-py2.py3-none-any.whl (105 KB)
2.0.2	2019-12-11
tokenizer-2.0.2-py2.py3-none-any.whl (105 KB)
2.0.1	2019-12-09
tokenizer-2.0.1-py2.py3-none-any.whl (105 KB)
2.0.0	2019-12-04
tokenizer-2.0.0-py2.py3-none-any.whl (104 KB)
1.4.1	2019-10-22
tokenizer-1.4.1-py2.py3-none-any.whl (78 KB)
1.4.0	2019-07-16
tokenizer-1.4.0-py2.py3-none-any.whl (76 KB)
1.3.0	2019-05-21
tokenizer-1.3.0-py2.py3-none-any.whl (74 KB)
1.2.3	2019-05-03
tokenizer-1.2.3-py2.py3-none-any.whl (71 KB)
1.2.2	2019-04-26
tokenizer-1.2.2-py2.py3-none-any.whl (63 KB)
1.2.1	2019-02-18
tokenizer-1.2.1-py2.py3-none-any.whl (62 KB)
1.2.0	2019-02-07
tokenizer-1.2.0-py2.py3-none-any.whl (62 KB)
1.1.2	2019-01-10
tokenizer-1.1.2-py2.py3-none-any.whl (61 KB)
1.1.1	2019-01-04
tokenizer-1.1.1-py2.py3-none-any.whl (61 KB)
1.1.0	2019-01-02
tokenizer-1.1.0-py2.py3-none-any.whl (60 KB)
1.0.9	2018-12-29
tokenizer-1.0.9-py2.py3-none-any.whl (60 KB)
1.0.8	2018-10-30
tokenizer-1.0.8-py2.py3-none-any.whl (59 KB)
1.0.7	2018-09-25
tokenizer-1.0.7-py2.py3-none-any.whl (67 KB)
1.0.6	2018-08-23
tokenizer-1.0.6-py2.py3-none-any.whl (67 KB)
1.0.5	2018-07-23
tokenizer-1.0.5-py2.py3-none-any.whl (59 KB)
1.0.4	2018-06-08
tokenizer-1.0.4-py2.py3-none-any.whl (54 KB)
1.0.3	2018-04-30
tokenizer-1.0.3-py2.py3-none-any.whl (44 KB)
1.0.2	2018-04-25
tokenizer-1.0.2-py2.py3-none-any.whl (44 KB)
1.0.1	2018-04-24
tokenizer-1.0.1-py2.py3-none-any.whl (44 KB)
1.0.0	2018-04-20
tokenizer-1.0.0-py2.py3-none-any.whl (43 KB)
0.1.14	2018-04-11
tokenizer-0.1.14-py2.py3-none-any.whl (43 KB)
0.1.12	2018-04-05
tokenizer-0.1.12-py2.py3-none-any.whl (43 KB)
0.1.11	2018-03-27
tokenizer-0.1.11-py2.py3-none-any.whl (43 KB)
0.1.10	2018-03-13
tokenizer-0.1.10-py2.py3-none-any.whl (56 KB)
0.1.3	2017-10-01
tokenizer-0.1.3-py2.py3-none-any.whl (57 KB)
0.1.2	2017-10-01
tokenizer-0.1.2-py3-none-any.whl (57 KB)
0.1.1	2017-10-01
tokenizer-0.1.1-py3-none-any.whl (57 KB)

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue