piwheels

flexgen

Running large language models like OPT-175B/GPT-3 on a single GPU. Focusing on high-throughput large-batch generation.

Installation

In a virtualenv (see these instructions if you need to create one):

pip3 install flexgen

Dependencies

• tqdm
• transformers
• torch
• pulp
• attrs
• numpy

Releases

Version	Released	Bullseye Python 3.9	Bookworm Python 3.11	Trixie Python 3.13	Files
0.1.8	2023-10-09
flexgen-0.1.8-py3-none-any.whl (119 KB)
0.1.7	2023-03-01
flexgen-0.1.7-py3-none-any.whl (50 KB)
0.1.6	2023-02-27
flexgen-0.1.6-py3-none-any.whl (45 KB)
0.1.5	2023-02-26
flexgen-0.1.5-py3-none-any.whl (47 KB)
0.1.4	2023-02-26
flexgen-0.1.4-py3-none-any.whl (46 KB)
0.1.3	2023-02-26
flexgen-0.1.3-py3-none-any.whl (46 KB)
0.1.2	2023-02-26
flexgen-0.1.2-py3-none-any.whl (46 KB)
0.1.1	2023-02-26
flexgen-0.1.1-py3-none-any.whl (46 KB)

Issues with this package?

• Search issues for this package
• Package or version missing? Open a new issue
• Something else? Open a new issue