piwheels

swebench

The official SWE-bench package - a benchmark for evaluating LMs on software engineering

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install swebench

Dependencies

• tqdm
• unidiff
• python-dotenv
• rich
• beautifulsoup4
• docker
• datasets
• ghapi
• modal
• chardet
• gitpython
• pre-commit
• tenacity
• requests

Releases

Version	Released	Bullseye Python 3.9	Bookworm Python 3.11	Trixie Python 3.13	Files
4.1.0	2025-09-11
swebench-4.1.0-py3-none-any.whl (154 KB)
4.0.5	2025-09-03
swebench-4.0.5-py3-none-any.whl (152 KB)
4.0.4	2025-07-17
swebench-4.0.4-py3-none-any.whl (152 KB)
4.0.3	2025-05-07
swebench-4.0.3-py3-none-any.whl (151 KB)
4.0.2	2025-05-07
swebench-4.0.2-py3-none-any.whl (8 KB)
4.0.2.dev0 pre-release	2025-05-07
swebench-4.0.2.dev0-py3-none-any.whl (151 KB)
4.0.1	2025-05-07
swebench-4.0.1-py3-none-any.whl (8 KB)
4.0.0	2025-05-07
swebench-4.0.0-py3-none-any.whl (8 KB)
3.0.17	2025-04-18
swebench-3.0.17-py3-none-any.whl (123 KB)
3.0.16	2025-04-09
swebench-3.0.16-py3-none-any.whl (123 KB)
3.0.15	2025-03-02
swebench-3.0.15-py3-none-any.whl (122 KB)
3.0.14	2025-03-01
swebench-3.0.14-py3-none-any.whl (122 KB)
3.0.13	2025-02-24
swebench-3.0.13-py3-none-any.whl (123 KB)
3.0.12	2025-02-19
swebench-3.0.12-py3-none-any.whl (123 KB)
3.0.11	2025-02-19
swebench-3.0.11-py3-none-any.whl (123 KB)
3.0.10	2025-02-19
swebench-3.0.10-py3-none-any.whl (123 KB)
3.0.9	2025-02-19
swebench-3.0.9-py3-none-any.whl (123 KB)
3.0.8	2025-02-12
swebench-3.0.8-py3-none-any.whl (122 KB)
3.0.7	2025-02-11
swebench-3.0.7-py3-none-any.whl (122 KB)
3.0.6	2025-02-03
swebench-3.0.6-py3-none-any.whl (122 KB)
3.0.5	2025-02-01
swebench-3.0.5-py3-none-any.whl (121 KB)
3.0.4	2025-01-21
swebench-3.0.4-py3-none-any.whl (121 KB)
3.0.3	2025-01-17
swebench-3.0.3-py3-none-any.whl (121 KB)
3.0.2	2025-01-16
swebench-3.0.2-py3-none-any.whl (121 KB)
3.0.1	2025-01-13
swebench-3.0.1-py3-none-any.whl (121 KB)
3.0.0	2025-01-13
swebench-3.0.0-py3-none-any.whl (113 KB)
2.1.8	2025-01-11
swebench-2.1.8-py3-none-any.whl (108 KB)
2.1.7	2024-12-10
swebench-2.1.7-py3-none-any.whl (101 KB)
2.1.6	2024-11-24
swebench-2.1.6-py3-none-any.whl (101 KB)
2.1.5	2024-11-24
swebench-2.1.5-py3-none-any.whl (101 KB)
2.1.4	2024-11-22
swebench-2.1.4-py3-none-any.whl (100 KB)
2.1.3	2024-11-17
swebench-2.1.3-py3-none-any.whl (100 KB)
2.1.2	2024-11-11
swebench-2.1.2-py3-none-any.whl (100 KB)
2.1.1	2024-10-29
swebench-2.1.1-py3-none-any.whl (100 KB)
2.1.0	2024-10-08
swebench-2.1.0-py3-none-any.whl (100 KB)
2.0.13	2024-08-20
swebench-2.0.13-py3-none-any.whl (100 KB)
2.0.12	2024-07-22
swebench-2.0.12-py3-none-any.whl (99 KB)
2.0.11	2024-07-15
swebench-2.0.11-py3-none-any.whl (99 KB)
2.0.10	2024-07-15
swebench-2.0.10-py3-none-any.whl (99 KB)
2.0.9	2024-07-10
swebench-2.0.9-py3-none-any.whl (99 KB)
2.0.8	2024-07-08
swebench-2.0.8-py3-none-any.whl (98 KB)
2.0.7	2024-07-08
swebench-2.0.7-py3-none-any.whl (99 KB)
2.0.6	2024-07-08
swebench-2.0.6-py3-none-any.whl (99 KB)
2.0.5	2024-07-08
swebench-2.0.5-py3-none-any.whl (99 KB)
2.0.4	2024-07-05
swebench-2.0.4-py3-none-any.whl (98 KB)
2.0.3	2024-07-02
swebench-2.0.3-py3-none-any.whl (98 KB)
2.0.2	2024-06-27
swebench-2.0.2-py3-none-any.whl (98 KB)
2.0.1	2024-06-27
swebench-2.0.1-py3-none-any.whl (98 KB)
2.0.0	2024-06-27
swebench-2.0.0-py3-none-any.whl (97 KB)
1.1.5	2024-05-17
swebench-1.1.5-py3-none-any.whl (96 KB)
1.1.4	2024-05-16
swebench-1.1.4-py3-none-any.whl (96 KB)
1.1.3	2024-05-16
swebench-1.1.3-py3-none-any.whl (96 KB)
1.1.2	2024-05-16
swebench-1.1.2-py3-none-any.whl (96 KB)
1.1.1	2024-05-15
swebench-1.1.1-py3-none-any.whl (96 KB)
1.1.0	2024-04-15
swebench-1.1.0-py3-none-any.whl (96 KB)
1.0.21	2024-04-12
swebench-1.0.21-py3-none-any.whl (96 KB)
1.0.20	2024-04-12
swebench-1.0.20-py3-none-any.whl (96 KB)
1.0.19	2024-04-11
swebench-1.0.19-py3-none-any.whl (96 KB)
1.0.18	2024-04-11
swebench-1.0.18-py3-none-any.whl (96 KB)
1.0.17	2024-04-11
swebench-1.0.17-py3-none-any.whl (96 KB)
1.0.16	2024-04-11
swebench-1.0.16-py3-none-any.whl (95 KB)
1.0.15	2024-04-11
swebench-1.0.15-py3-none-any.whl (95 KB)
1.0.14	2024-04-11
swebench-1.0.14-py3-none-any.whl (95 KB)
1.0.13	2024-04-11
swebench-1.0.13-py3-none-any.whl (95 KB)
1.0.12	2024-04-10
swebench-1.0.12-py3-none-any.whl (95 KB)
1.0.11	2024-04-10
swebench-1.0.11-py3-none-any.whl (95 KB)
1.0.10	2024-04-10
swebench-1.0.10-py3-none-any.whl (95 KB)
1.0.9	2024-04-10
swebench-1.0.9-py3-none-any.whl (95 KB)
1.0.8	2024-04-09
swebench-1.0.8-py3-none-any.whl (95 KB)
1.0.7	2024-04-09
swebench-1.0.7-py3-none-any.whl (95 KB)
1.0.6	2024-04-09
swebench-1.0.6-py3-none-any.whl (95 KB)
1.0.5	2024-04-07
swebench-1.0.5-py3-none-any.whl (95 KB)
1.0.4	2024-04-05
swebench-1.0.4-py3-none-any.whl (94 KB)
1.0.3	2024-04-04
swebench-1.0.3-py3-none-any.whl (94 KB)
1.0.2	2024-04-02
swebench-1.0.2-py3-none-any.whl (94 KB)
1.0.1	2024-04-01
swebench-1.0.1-py3-none-any.whl (93 KB)
1.0.0	2024-04-01
swebench-1.0.0-py3-none-any.whl (93 KB)
0.6.9.2	2024-04-01
swebench-0.6.9.2-py3-none-any.whl (51 KB)
0.6.9.1	2024-04-01
swebench-0.6.9.1-py3-none-any.whl (6 KB)
0.6.9	2024-04-01
swebench-0.6.9-py3-none-any.whl (6 KB)
0.6.8.4	2024-03-28
swebench-0.6.8.4-py3-none-any.whl (92 KB)
0.6.8.3	2024-03-28
swebench-0.6.8.3-py3-none-any.whl (92 KB)
0.6.8.2	2024-03-28
swebench-0.6.8.2-py3-none-any.whl (45 KB)
0.6.8.1	2024-03-28
swebench-0.6.8.1-py3-none-any.whl (45 KB)
0.6.8	2024-03-21
swebench-0.6.8-py3-none-any.whl (48 KB)
0.6.7	2024-03-17
swebench-0.6.7-py3-none-any.whl (47 KB)
0.6.6	2024-03-17
swebench-0.6.6-py3-none-any.whl (47 KB)
0.6.5	2024-03-17
swebench-0.6.5-py3-none-any.whl (47 KB)
0.6.4	2024-03-17
swebench-0.6.4-py3-none-any.whl (47 KB)
0.6.3	2024-03-17
swebench-0.6.3-py3-none-any.whl (47 KB)
0.6.2	2024-03-17
swebench-0.6.2-py3-none-any.whl (47 KB)
0.6.1	2024-03-14
swebench-0.6.1-py3-none-any.whl (47 KB)
0.6	2024-02-28
swebench-0.6-py3-none-any.whl (46 KB)
0.5.9	2024-01-25
swebench-0.5.9-py3-none-any.whl (47 KB)
0.5.8	2024-01-15
swebench-0.5.8-py3-none-any.whl (47 KB)
0.5.7	2023-12-20
swebench-0.5.7-py3-none-any.whl (46 KB)
0.5.6	2023-12-19
swebench-0.5.6-py3-none-any.whl (46 KB)
0.5.5	2023-11-28
swebench-0.5.5-py3-none-any.whl (46 KB)
0.5.4	2023-11-28
swebench-0.5.4-py3-none-any.whl (46 KB)
0.5.3	2023-11-28
swebench-0.5.3-py3-none-any.whl (46 KB)
0.5.2	2023-11-28
swebench-0.5.2-py3-none-any.whl (46 KB)
0.5.1	2023-11-28
swebench-0.5.1-py3-none-any.whl (46 KB)
0.5.0	2023-11-28
swebench-0.5.0-py3-none-any.whl (46 KB)
0.4.9	2023-11-28
swebench-0.4.9-py3-none-any.whl (46 KB)
0.4.8	2023-11-28
swebench-0.4.8-py3-none-any.whl (45 KB)
0.4.7	2023-11-20
swebench-0.4.7-py3-none-any.whl (45 KB)
0.4.6	2023-11-15
swebench-0.4.6-py3-none-any.whl (55 KB)
0.4.5	2023-11-08
swebench-0.4.5-py3-none-any.whl (45 KB)
0.4.4	2023-11-08
swebench-0.4.4-py3-none-any.whl (45 KB)
0.4.3	2023-11-05
swebench-0.4.3-py3-none-any.whl (45 KB)
0.4.2	2023-11-05
swebench-0.4.2-py3-none-any.whl (44 KB)
0.4.1	2023-11-03
swebench-0.4.1-py3-none-any.whl (44 KB)
0.3	2023-11-01
0.2.1	2023-11-01
0.2	2023-11-01
0.1	2023-11-01

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue