2025-11-13T11:59:57,845 Created temporary directory: /tmp/pip-ephem-wheel-cache-jau7hfib 2025-11-13T11:59:57,847 Created temporary directory: /tmp/pip-build-tracker-d9di2q_b 2025-11-13T11:59:57,847 Initialized build tracking at /tmp/pip-build-tracker-d9di2q_b 2025-11-13T11:59:57,848 Created build tracker: /tmp/pip-build-tracker-d9di2q_b 2025-11-13T11:59:57,848 Entered build tracker: /tmp/pip-build-tracker-d9di2q_b 2025-11-13T11:59:57,849 Created temporary directory: /tmp/pip-wheel-3fnn0vsq 2025-11-13T11:59:57,852 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2025-11-13T11:59:57,854 Created temporary directory: /tmp/pip-ephem-wheel-cache-k2d737ne 2025-11-13T11:59:57,875 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2025-11-13T11:59:57,879 2 location(s) to search for versions of lighteval: 2025-11-13T11:59:57,879 * https://pypi.org/simple/lighteval/ 2025-11-13T11:59:57,879 * https://www.piwheels.org/simple/lighteval/ 2025-11-13T11:59:57,880 Fetching project page and analyzing links: https://pypi.org/simple/lighteval/ 2025-11-13T11:59:57,880 Getting page https://pypi.org/simple/lighteval/ 2025-11-13T11:59:57,882 Found index url https://pypi.org/simple 2025-11-13T11:59:58,097 Fetched page https://pypi.org/simple/lighteval/ as application/vnd.pypi.simple.v1+json 2025-11-13T11:59:58,105 Found link https://files.pythonhosted.org/packages/91/70/3a0c520287b432278921dafe9b4b20e4d639ecb62a1cdb8dfc14a7cf1122/lighteval-0.0.1.tar.gz (from https://pypi.org/simple/lighteval/), version: 0.0.1 2025-11-13T11:59:58,106 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/83/b0/7dbdeffab3e2d3a0791f1747818c65c3e76e4c343593712aaaee990bea93/lighteval-0.0.2-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,107 Found link https://files.pythonhosted.org/packages/b5/e9/30eaccb5bfb4ca9802bbc3b105b7c8888bdd9bff72d7a7b85ffcb6cd4ad1/lighteval-0.0.2.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.0.2 2025-11-13T11:59:58,108 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/ce/22/8e9fb7ef899f757df0eafed9e29fd8db9d4c694a3ba7e276bedc5c8a4dc3/lighteval-0.1.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,109 Found link https://files.pythonhosted.org/packages/72/c3/07b4cb157f8fa1c0465c32f505418de1df659c67b2bbfa543c5bc9ff9dde/lighteval-0.1.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.1.0 2025-11-13T11:59:58,110 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/af/d8/92cee84f102f5c801240e90e6380a1f2958aa8ae977fe3f7a6b9804214a0/lighteval-0.1.1.dev0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,111 Found link https://files.pythonhosted.org/packages/73/b2/0b510938a01164061a56b871af8b138f4ade7fdea6bb96ed5180e345ba2a/lighteval-0.1.1.dev0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.1.1.dev0 2025-11-13T11:59:58,112 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/a4/f7/9385103352347eaa2500b3060929a2242143db19c937eb56e488753a4d99/lighteval-0.1.1-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,113 Found link https://files.pythonhosted.org/packages/aa/10/97c3a28e599d930aefaa173edc596980df23dc593883a5b0a3ad00c0e2f5/lighteval-0.1.1.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.1.1 2025-11-13T11:59:58,113 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/51/0b/304021094a01fbc94a98b7729b2044106475b23286031083fa9217fea981/lighteval-0.2.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,114 Found link https://files.pythonhosted.org/packages/c2/b3/dc97dfbdd406d8ce4a7f50d86ef5a24d561e4bfb649783da0807f7cc6cb8/lighteval-0.2.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.2.0 2025-11-13T11:59:58,115 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/14/e2/4c1dc89e6a6c71c6d4745ffd1657e57ca4381dfc48d2c2af6d38215f9f01/lighteval-0.3.0a0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,116 Found link https://files.pythonhosted.org/packages/e9/5f/9bff4b2f649b9838411bf5a3e29b8eaef0287baee2af56419c7845f43a25/lighteval-0.3.0a0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.3.0a0 2025-11-13T11:59:58,117 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/f5/e7/33f933d80fe537dacdf2677df18437ba8b90e6228936564dd246c295809a/lighteval-0.3.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,118 Found link https://files.pythonhosted.org/packages/02/d7/8c52ae664c106385547c5e895b8614b58eb97e1a721ccee0e278739bd0ff/lighteval-0.3.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.3.0 2025-11-13T11:59:58,118 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/e9/b4/27c6f424b053b66cec7c66dd57823089d5349164dcc82a965f1b230831c5/lighteval-0.4.0a0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,119 Found link https://files.pythonhosted.org/packages/a1/9d/52971d034e2fda19bee1f31bbc8e8683eee2a4807b530e3d2fc50215ef37/lighteval-0.4.0a0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.4.0a0 2025-11-13T11:59:58,120 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/c5/3e/ceea138f6702f5dd010544b75606ba2f51a48ae0bbcb3fb1051216beb68e/lighteval-0.5.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,121 Found link https://files.pythonhosted.org/packages/b9/72/2cbd211318e1dd4ab6653cfe96672dfd68d17a02e762f09b14e8cd7bb6f3/lighteval-0.5.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.5.0 2025-11-13T11:59:58,121 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/ba/f8/c3f757064572b62ef63cb8e45e093245b0327e9f8e1c93f2aa57e227a33e/lighteval-0.6.2-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,122 Found link https://files.pythonhosted.org/packages/65/dd/547f2af88bc4c56ce19123f66701cdc5762f33cc9b49bedb19ae03b26fcd/lighteval-0.6.2.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.6.2 2025-11-13T11:59:58,123 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/86/d8/655d36dcee8f56dae06321448da2d19c8c780051fa5b171f35fae91f7e18/lighteval-0.7.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,124 Found link https://files.pythonhosted.org/packages/65/e1/bd70449c07c6b4259cc8395ab3c8cc6e433a920084bbe12fe651bc3480d0/lighteval-0.7.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.7.0 2025-11-13T11:59:58,125 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/52/88/8c120913b07a1a828b7c2874055a022e2ce098327736ca7602d14d323ad7/lighteval-0.8.1-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,126 Found link https://files.pythonhosted.org/packages/11/4d/9387d49759cc02c9f546ec7e9975a19e88ff573170823153375e58a34c4a/lighteval-0.8.1.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.8.1 2025-11-13T11:59:58,126 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/34/fe/4a5ddcc73f9a8f320c47a1c65f497f8c2d8733730e621407536bba612e6f/lighteval-0.9.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,128 Found link https://files.pythonhosted.org/packages/a3/0c/1d849e968023263f4b96952aa04a11c522eb8ad241262c1705e9a13f1762/lighteval-0.9.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.9.0 2025-11-13T11:59:58,128 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/7a/1c/3bbd71a3c2402661eeda4417202cccd832105d92454c1fa531ef07b4d7ec/lighteval-0.9.1-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,129 Found link https://files.pythonhosted.org/packages/2d/dc/6a41ed7d2eb4c09c607138c97fae2fdb64db5d1b9e9e198da061e8894051/lighteval-0.9.1.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.9.1 2025-11-13T11:59:58,130 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/ae/d6/ee335f2524642a7443cc113da0101d350273d42ba4eac7f637bd44db5d0e/lighteval-0.9.2-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,131 Found link https://files.pythonhosted.org/packages/2e/2c/822126ca365cde3cedc91e9e5beaf6d33168ce18261f982686d1c4447670/lighteval-0.9.2.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.9.2 2025-11-13T11:59:58,131 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/7c/8e/d48315622c38086b94d8274edc5c3630be03befcce36a5ef318e0d278f0b/lighteval-0.10.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,132 Found link https://files.pythonhosted.org/packages/e9/2c/8232d77ba5c22342a21ed313556a8cc998e63c5ff10b2f87947527822d8b/lighteval-0.10.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.10.0 2025-11-13T11:59:58,133 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/d3/ba/bf0a54eb1969cae565703addcecb923d19aa7d2bccc6182f388c66b6980b/lighteval-0.11.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,134 Found link https://files.pythonhosted.org/packages/e6/a2/50e2251012de8587442d39485946bc2e579bb63551757389c60cd20ce6cd/lighteval-0.11.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.11.0 2025-11-13T11:59:58,135 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/d6/34/4b438b81badd48672b8e72e721d0bdcdee0a6c8620f74362d8f26228571f/lighteval-0.12.0-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,136 Found link https://files.pythonhosted.org/packages/f8/1a/e0546a2e5786d0d1c5477361202dba01762de51dbfb4569ef438bb291fe5/lighteval-0.12.0.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.12.0 2025-11-13T11:59:58,136 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/0c/6b/65e23781dd936db292c9b558978ca57e873b206254af1d3fcd60a78ca1b4/lighteval-0.12.1-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,137 Found link https://files.pythonhosted.org/packages/08/25/196b6b4e3b1358faa79aca54100e107824c9bb3ee09aa234b1da4b83843b/lighteval-0.12.1.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.12.1 2025-11-13T11:59:58,138 Skipping link: No binaries permitted for lighteval: https://files.pythonhosted.org/packages/8e/4f/5efe2fdfd4a39c3654136748bdb229393e2a39f44421ef6c16b634be8f10/lighteval-0.12.2-py3-none-any.whl (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,139 Found link https://files.pythonhosted.org/packages/23/ee/8a24f8c29aa5d06eb175089b3815cd8cac251ae5c3ce5f622c7ff8548a4b/lighteval-0.12.2.tar.gz (from https://pypi.org/simple/lighteval/) (requires-python:>=3.10), version: 0.12.2 2025-11-13T11:59:58,140 Fetching project page and analyzing links: https://www.piwheels.org/simple/lighteval/ 2025-11-13T11:59:58,141 Getting page https://www.piwheels.org/simple/lighteval/ 2025-11-13T11:59:58,142 Found index url https://www.piwheels.org/simple 2025-11-13T11:59:58,345 Fetched page https://www.piwheels.org/simple/lighteval/ as text/html 2025-11-13T11:59:58,351 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.12.1-py3-none-any.whl#sha256=a489e86789e9c16786dc62dce35176be38b722dc05cbde1b9570456958fb8395 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,352 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.12.0-py3-none-any.whl#sha256=9232b4db5aab006a51962753e95165ccb057407597936804be72bd24531b24b3 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,352 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.11.0-py3-none-any.whl#sha256=6ce0c089e9d8bb70a3a9567692de9197b27f7de1f867ca89ebc209225fe3fa86 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,353 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.10.0-py3-none-any.whl#sha256=984e54961dfab22f14490603e818bd5fc52f7a9cb7325167cd4c722cd3a617c8 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,353 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.9.2-py3-none-any.whl#sha256=524551d8382e05b833a815efaa0c3d72e6f8394f779f7d0996c6bd32480c8bb3 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,354 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.9.1-py3-none-any.whl#sha256=547617dc5562604712e54b11110df507fc7bb93dbe003d95bd5d8f1b619fc5e5 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,354 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.9.0-py3-none-any.whl#sha256=f08fa34420204de0e093b908f9993b7857a704f1ea3ed352eb09ef2c821e7205 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,355 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.8.1-py3-none-any.whl#sha256=fc457974bc07ba6ec3f5f1c3ad74772add7907118d341df8bbf833064d9d1546 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,356 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.7.0-py3-none-any.whl#sha256=c2a51aca3df46874ddc37983dabd50cc525b25fb2b3c7eab15e445a9f0b9b712 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,356 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.6.2-py3-none-any.whl#sha256=dbbe93471a2dd20cdc138ee008b29e05bb7011af99679649aee415fdc38374e4 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,357 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.5.0-py3-none-any.whl#sha256=62bf9ced8a1139ca7c79da092c98d6b4d70d27a4007d8cdfde70f64eb9b98da2 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,358 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.4.0a0-py3-none-any.whl#sha256=8694039d9a42ffe2c54d5bc4d72d4a7ab3bf1b4032c970eb52fa05455da4d6a1 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,358 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.3.0-py3-none-any.whl#sha256=278613752d9402caf51c2a009228c2c91bb801a71dadc88586c41a764477da7a (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,359 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.3.0a0-py3-none-any.whl#sha256=ca834b38d37c41993d1573606b9f64d28e1218494118389f552f4fbc5c7fe4fd (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,360 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.2.0-py3-none-any.whl#sha256=f4dfdbe773e3f0a3958dddfd65b958a347095d693191d5c453d00de9f30344ae (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,360 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.1.1-py3-none-any.whl#sha256=296c95ef71b3014c30e8fe406bb95469a3d484c721fa61efbb1fe99dce67f950 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,361 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.1.1.dev0-py3-none-any.whl#sha256=e96ff26cffee66313bad7d96cc3522f90457ccc3fa292862034af2db96f4ef73 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,361 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.1.0-py3-none-any.whl#sha256=780a9fdd33d3b41060347f899486cd3ffab6f44f46af6c5d9a18ff10e725731d (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,362 Skipping link: No binaries permitted for lighteval: https://archive1.piwheels.org/simple/lighteval/lighteval-0.0.2-py3-none-any.whl#sha256=08472d569e00511c2d1803932a9b21c03e4424d68970bf1e67d0ab490ef59019 (from https://www.piwheels.org/simple/lighteval/) (requires-python:>=3.10) 2025-11-13T11:59:58,362 Skipping link: not a file: https://www.piwheels.org/simple/lighteval/ 2025-11-13T11:59:58,363 Skipping link: not a file: https://pypi.org/simple/lighteval/ 2025-11-13T11:59:58,384 Given no hashes to check 1 links for project 'lighteval': discarding no candidates 2025-11-13T11:59:58,403 Collecting lighteval==0.12.2 2025-11-13T11:59:58,405 Created temporary directory: /tmp/pip-unpack-4dv3nra4 2025-11-13T11:59:58,538 Downloading lighteval-0.12.2.tar.gz (424 kB) 2025-11-13T11:59:59,220 Added lighteval==0.12.2 from https://files.pythonhosted.org/packages/23/ee/8a24f8c29aa5d06eb175089b3815cd8cac251ae5c3ce5f622c7ff8548a4b/lighteval-0.12.2.tar.gz to build tracker '/tmp/pip-build-tracker-d9di2q_b' 2025-11-13T11:59:59,226 Created temporary directory: /tmp/pip-build-env-_5jjhrf0 2025-11-13T11:59:59,231 Installing build dependencies: started 2025-11-13T11:59:59,232 Running command pip subprocess to install build dependencies 2025-11-13T12:00:00,354 Using pip 23.0.1 from /usr/lib/python3/dist-packages/pip (python 3.11) 2025-11-13T12:00:00,951 DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453 2025-11-13T12:00:00,975 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2025-11-13T12:00:02,703 Collecting setuptools>=61.0 2025-11-13T12:00:02,800 Using cached https://www.piwheels.org/simple/setuptools/setuptools-80.9.0-py3-none-any.whl (1.2 MB) 2025-11-13T12:00:05,695 Installing collected packages: setuptools 2025-11-13T12:00:09,427 Successfully installed setuptools-80.9.0 2025-11-13T12:00:09,692 Installing build dependencies: finished with status 'done' 2025-11-13T12:00:09,698 Getting requirements to build wheel: started 2025-11-13T12:00:09,699 Running command Getting requirements to build wheel 2025-11-13T12:00:10,357 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2025-11-13T12:00:10,357 !! 2025-11-13T12:00:10,359 ******************************************************************************** 2025-11-13T12:00:10,359 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2025-11-13T12:00:10,361 By 2026-Feb-18, you need to update your project and remove deprecated calls 2025-11-13T12:00:10,362 or your builds will no longer be supported. 2025-11-13T12:00:10,363 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:10,364 ******************************************************************************** 2025-11-13T12:00:10,366 !! 2025-11-13T12:00:10,366 corresp(dist, value, root_dir) 2025-11-13T12:00:10,459 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:10,459 !! 2025-11-13T12:00:10,460 ******************************************************************************** 2025-11-13T12:00:10,461 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:10,462 License :: OSI Approved :: MIT License 2025-11-13T12:00:10,463 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:10,464 ******************************************************************************** 2025-11-13T12:00:10,465 !! 2025-11-13T12:00:10,466 dist._finalize_license_expression() 2025-11-13T12:00:10,467 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:10,467 !! 2025-11-13T12:00:10,469 ******************************************************************************** 2025-11-13T12:00:10,469 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:10,470 License :: OSI Approved :: MIT License 2025-11-13T12:00:10,472 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:10,472 ******************************************************************************** 2025-11-13T12:00:10,473 !! 2025-11-13T12:00:10,474 self._finalize_license_expression() 2025-11-13T12:00:10,474 running egg_info 2025-11-13T12:00:10,481 writing src/lighteval.egg-info/PKG-INFO 2025-11-13T12:00:10,499 writing dependency_links to src/lighteval.egg-info/dependency_links.txt 2025-11-13T12:00:10,501 writing entry points to src/lighteval.egg-info/entry_points.txt 2025-11-13T12:00:10,511 writing requirements to src/lighteval.egg-info/requires.txt 2025-11-13T12:00:10,513 writing top-level names to src/lighteval.egg-info/top_level.txt 2025-11-13T12:00:10,576 reading manifest file 'src/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:10,592 reading manifest template 'MANIFEST.in' 2025-11-13T12:00:10,593 warning: no files found matching 'src/lighteval/tasks/tasks_table.jsonl' 2025-11-13T12:00:10,594 warning: no files found matching 'src/lighteval/metrics/*.jsonl' 2025-11-13T12:00:10,595 adding license file 'LICENSE' 2025-11-13T12:00:10,613 writing manifest file 'src/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:10,727 Getting requirements to build wheel: finished with status 'done' 2025-11-13T12:00:10,730 Created temporary directory: /tmp/pip-modern-metadata-__vahaza 2025-11-13T12:00:10,733 Preparing metadata (pyproject.toml): started 2025-11-13T12:00:10,734 Running command Preparing metadata (pyproject.toml) 2025-11-13T12:00:11,336 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2025-11-13T12:00:11,336 !! 2025-11-13T12:00:11,337 ******************************************************************************** 2025-11-13T12:00:11,338 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2025-11-13T12:00:11,339 By 2026-Feb-18, you need to update your project and remove deprecated calls 2025-11-13T12:00:11,340 or your builds will no longer be supported. 2025-11-13T12:00:11,340 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:11,341 ******************************************************************************** 2025-11-13T12:00:11,342 !! 2025-11-13T12:00:11,343 corresp(dist, value, root_dir) 2025-11-13T12:00:11,458 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:11,459 !! 2025-11-13T12:00:11,460 ******************************************************************************** 2025-11-13T12:00:11,461 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:11,462 License :: OSI Approved :: MIT License 2025-11-13T12:00:11,464 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:11,464 ******************************************************************************** 2025-11-13T12:00:11,466 !! 2025-11-13T12:00:11,466 dist._finalize_license_expression() 2025-11-13T12:00:11,472 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:11,473 !! 2025-11-13T12:00:11,475 ******************************************************************************** 2025-11-13T12:00:11,475 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:11,477 License :: OSI Approved :: MIT License 2025-11-13T12:00:11,478 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:11,479 ******************************************************************************** 2025-11-13T12:00:11,480 !! 2025-11-13T12:00:11,480 self._finalize_license_expression() 2025-11-13T12:00:11,486 running dist_info 2025-11-13T12:00:11,505 creating /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info 2025-11-13T12:00:11,507 writing /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/PKG-INFO 2025-11-13T12:00:11,528 writing dependency_links to /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/dependency_links.txt 2025-11-13T12:00:11,531 writing entry points to /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/entry_points.txt 2025-11-13T12:00:11,543 writing requirements to /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/requires.txt 2025-11-13T12:00:11,544 writing top-level names to /tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/top_level.txt 2025-11-13T12:00:11,547 writing manifest file '/tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:11,608 reading manifest file '/tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:11,612 reading manifest template 'MANIFEST.in' 2025-11-13T12:00:11,613 warning: no files found matching 'src/lighteval/tasks/tasks_table.jsonl' 2025-11-13T12:00:11,614 warning: no files found matching 'src/lighteval/metrics/*.jsonl' 2025-11-13T12:00:11,615 adding license file 'LICENSE' 2025-11-13T12:00:11,626 writing manifest file '/tmp/pip-modern-metadata-__vahaza/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:11,627 creating '/tmp/pip-modern-metadata-__vahaza/lighteval-0.12.2.dist-info' 2025-11-13T12:00:11,779 Preparing metadata (pyproject.toml): finished with status 'done' 2025-11-13T12:00:11,785 Source in /tmp/pip-wheel-3fnn0vsq/lighteval_4456cbcaa4494a658ecb1dca4dabb92c has version 0.12.2, which satisfies requirement lighteval==0.12.2 from https://files.pythonhosted.org/packages/23/ee/8a24f8c29aa5d06eb175089b3815cd8cac251ae5c3ce5f622c7ff8548a4b/lighteval-0.12.2.tar.gz 2025-11-13T12:00:11,786 Removed lighteval==0.12.2 from https://files.pythonhosted.org/packages/23/ee/8a24f8c29aa5d06eb175089b3815cd8cac251ae5c3ce5f622c7ff8548a4b/lighteval-0.12.2.tar.gz from build tracker '/tmp/pip-build-tracker-d9di2q_b' 2025-11-13T12:00:11,794 Created temporary directory: /tmp/pip-unpack-ah4oe_p7 2025-11-13T12:00:11,794 Building wheels for collected packages: lighteval 2025-11-13T12:00:11,799 Created temporary directory: /tmp/pip-wheel-ro5n5phe 2025-11-13T12:00:11,799 Destination directory: /tmp/pip-wheel-ro5n5phe 2025-11-13T12:00:11,802 Building wheel for lighteval (pyproject.toml): started 2025-11-13T12:00:11,803 Running command Building wheel for lighteval (pyproject.toml) 2025-11-13T12:00:12,369 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated 2025-11-13T12:00:12,369 !! 2025-11-13T12:00:12,370 ******************************************************************************** 2025-11-13T12:00:12,371 Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0). 2025-11-13T12:00:12,372 By 2026-Feb-18, you need to update your project and remove deprecated calls 2025-11-13T12:00:12,372 or your builds will no longer be supported. 2025-11-13T12:00:12,373 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:12,374 ******************************************************************************** 2025-11-13T12:00:12,375 !! 2025-11-13T12:00:12,375 corresp(dist, value, root_dir) 2025-11-13T12:00:12,459 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:12,459 !! 2025-11-13T12:00:12,460 ******************************************************************************** 2025-11-13T12:00:12,461 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:12,462 License :: OSI Approved :: MIT License 2025-11-13T12:00:12,463 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:12,463 ******************************************************************************** 2025-11-13T12:00:12,465 !! 2025-11-13T12:00:12,465 dist._finalize_license_expression() 2025-11-13T12:00:12,469 /tmp/pip-build-env-_5jjhrf0/overlay/local/lib/python3.11/dist-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated. 2025-11-13T12:00:12,470 !! 2025-11-13T12:00:12,471 ******************************************************************************** 2025-11-13T12:00:12,472 Please consider removing the following classifiers in favor of a SPDX license expression: 2025-11-13T12:00:12,473 License :: OSI Approved :: MIT License 2025-11-13T12:00:12,474 See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details. 2025-11-13T12:00:12,474 ******************************************************************************** 2025-11-13T12:00:12,475 !! 2025-11-13T12:00:12,475 self._finalize_license_expression() 2025-11-13T12:00:12,476 running bdist_wheel 2025-11-13T12:00:12,495 running build 2025-11-13T12:00:12,496 running build_py 2025-11-13T12:00:12,503 creating build/lib/lighteval 2025-11-13T12:00:12,506 copying src/lighteval/main_accelerate.py -> build/lib/lighteval 2025-11-13T12:00:12,508 copying src/lighteval/main_tasks.py -> build/lib/lighteval 2025-11-13T12:00:12,511 copying src/lighteval/cli_args.py -> build/lib/lighteval 2025-11-13T12:00:12,513 copying src/lighteval/main_nanotron.py -> build/lib/lighteval 2025-11-13T12:00:12,516 copying src/lighteval/__init__.py -> build/lib/lighteval 2025-11-13T12:00:12,518 copying src/lighteval/data.py -> build/lib/lighteval 2025-11-13T12:00:12,521 copying src/lighteval/main_inspect.py -> build/lib/lighteval 2025-11-13T12:00:12,524 copying src/lighteval/__main__.py -> build/lib/lighteval 2025-11-13T12:00:12,526 copying src/lighteval/main_sglang.py -> build/lib/lighteval 2025-11-13T12:00:12,529 copying src/lighteval/pipeline.py -> build/lib/lighteval 2025-11-13T12:00:12,531 copying src/lighteval/main_vllm.py -> build/lib/lighteval 2025-11-13T12:00:12,534 copying src/lighteval/main_endpoint.py -> build/lib/lighteval 2025-11-13T12:00:12,536 copying src/lighteval/main_custom.py -> build/lib/lighteval 2025-11-13T12:00:12,538 copying src/lighteval/main_baseline.py -> build/lib/lighteval 2025-11-13T12:00:12,541 creating build/lib/lighteval/metrics 2025-11-13T12:00:12,542 copying src/lighteval/metrics/__init__.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,544 copying src/lighteval/metrics/metrics_corpus.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,547 copying src/lighteval/metrics/metrics_sample.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,550 copying src/lighteval/metrics/metrics.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,552 copying src/lighteval/metrics/normalizations.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,555 copying src/lighteval/metrics/dynamic_metrics.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,557 copying src/lighteval/metrics/sample_preparator.py -> build/lib/lighteval/metrics 2025-11-13T12:00:12,560 creating build/lib/lighteval/logging 2025-11-13T12:00:12,561 copying src/lighteval/logging/evaluation_tracker.py -> build/lib/lighteval/logging 2025-11-13T12:00:12,564 copying src/lighteval/logging/info_loggers.py -> build/lib/lighteval/logging 2025-11-13T12:00:12,567 creating build/lib/lighteval/tasks 2025-11-13T12:00:12,568 copying src/lighteval/tasks/registry.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,571 copying src/lighteval/tasks/requests.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,573 copying src/lighteval/tasks/default_prompts.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,578 copying src/lighteval/tasks/__init__.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,580 copying src/lighteval/tasks/prompt_manager.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,582 copying src/lighteval/tasks/lighteval_task.py -> build/lib/lighteval/tasks 2025-11-13T12:00:12,585 creating build/lib/lighteval/models 2025-11-13T12:00:12,586 copying src/lighteval/models/model_output.py -> build/lib/lighteval/models 2025-11-13T12:00:12,588 copying src/lighteval/models/model_loader.py -> build/lib/lighteval/models 2025-11-13T12:00:12,590 copying src/lighteval/models/utils.py -> build/lib/lighteval/models 2025-11-13T12:00:12,593 copying src/lighteval/models/model_input.py -> build/lib/lighteval/models 2025-11-13T12:00:12,595 copying src/lighteval/models/abstract_model.py -> build/lib/lighteval/models 2025-11-13T12:00:12,598 creating build/lib/lighteval/utils 2025-11-13T12:00:12,599 copying src/lighteval/utils/__init__.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,601 copying src/lighteval/utils/cache_management.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,604 copying src/lighteval/utils/utils.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,606 copying src/lighteval/utils/language.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,608 copying src/lighteval/utils/timeout.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,610 copying src/lighteval/utils/parallelism.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,612 copying src/lighteval/utils/imports.py -> build/lib/lighteval/utils 2025-11-13T12:00:12,615 creating build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,616 copying src/lighteval/metrics/imports/__init__.py -> build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,618 copying src/lighteval/metrics/imports/data_stats_utils.py -> build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,620 copying src/lighteval/metrics/imports/bert_scorer.py -> build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,622 copying src/lighteval/metrics/imports/data_stats_metric.py -> build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,625 copying src/lighteval/metrics/imports/summac.py -> build/lib/lighteval/metrics/imports 2025-11-13T12:00:12,628 creating build/lib/lighteval/metrics/harness_compatibility 2025-11-13T12:00:12,629 copying src/lighteval/metrics/harness_compatibility/drop.py -> build/lib/lighteval/metrics/harness_compatibility 2025-11-13T12:00:12,631 copying src/lighteval/metrics/harness_compatibility/truthful_qa.py -> build/lib/lighteval/metrics/harness_compatibility 2025-11-13T12:00:12,634 creating build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,635 copying src/lighteval/metrics/utils/math_comparison.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,638 copying src/lighteval/metrics/utils/metric_utils.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,640 copying src/lighteval/metrics/utils/linguistic_tokenizers.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,642 copying src/lighteval/metrics/utils/judge_utils.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,645 copying src/lighteval/metrics/utils/extractive_match_utils.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,647 copying src/lighteval/metrics/utils/llm_as_judge.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,651 copying src/lighteval/metrics/utils/stderr.py -> build/lib/lighteval/metrics/utils 2025-11-13T12:00:12,655 creating build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,656 copying src/lighteval/tasks/tasks/real_toxicity_prompts.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,658 copying src/lighteval/tasks/tasks/xcopa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,661 copying src/lighteval/tasks/tasks/webqs.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,663 copying src/lighteval/tasks/tasks/entitymatching.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,665 copying src/lighteval/tasks/tasks/med_dialog.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,667 copying src/lighteval/tasks/tasks/arc.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,669 copying src/lighteval/tasks/tasks/legalsupport.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,670 copying src/lighteval/tasks/tasks/imdb.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,672 copying src/lighteval/tasks/tasks/mmlu.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,675 copying src/lighteval/tasks/tasks/wikitext.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,677 copying src/lighteval/tasks/tasks/unscramble.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,679 copying src/lighteval/tasks/tasks/babi_qa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,681 copying src/lighteval/tasks/tasks/logiqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,683 copying src/lighteval/tasks/tasks/the_pile.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,685 copying src/lighteval/tasks/tasks/truthfulqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,687 copying src/lighteval/tasks/tasks/math_500.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,689 copying src/lighteval/tasks/tasks/bigbench.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,693 copying src/lighteval/tasks/tasks/mathqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,695 copying src/lighteval/tasks/tasks/sacrebleu.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,698 copying src/lighteval/tasks/tasks/lexglue.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,700 copying src/lighteval/tasks/tasks/civil_comments.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,703 copying src/lighteval/tasks/tasks/hellaswag.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,705 copying src/lighteval/tasks/tasks/prost.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,706 copying src/lighteval/tasks/tasks/qasper.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,708 copying src/lighteval/tasks/tasks/gpqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,710 copying src/lighteval/tasks/tasks/summarization.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,712 copying src/lighteval/tasks/tasks/covid_dialogue.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,714 copying src/lighteval/tasks/tasks/natural_questions.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,716 copying src/lighteval/tasks/tasks/jeopardy.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,718 copying src/lighteval/tasks/tasks/pubmedqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,720 copying src/lighteval/tasks/tasks/slr_bench.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,721 copying src/lighteval/tasks/tasks/race_high.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,723 copying src/lighteval/tasks/tasks/storycloze.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,725 copying src/lighteval/tasks/tasks/coqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,727 copying src/lighteval/tasks/tasks/mmlu_redux.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,729 copying src/lighteval/tasks/tasks/aime.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,731 copying src/lighteval/tasks/tasks/bbq.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,734 copying src/lighteval/tasks/tasks/lsat_qa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,735 copying src/lighteval/tasks/tasks/squad_v2.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,737 copying src/lighteval/tasks/tasks/math.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,740 copying src/lighteval/tasks/tasks/headqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,741 copying src/lighteval/tasks/tasks/boolq.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,743 copying src/lighteval/tasks/tasks/blimp.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,747 copying src/lighteval/tasks/tasks/lambada.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,749 copying src/lighteval/tasks/tasks/piqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,751 copying src/lighteval/tasks/tasks/toxigen.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,753 copying src/lighteval/tasks/tasks/commonsenseqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,754 copying src/lighteval/tasks/tasks/glue.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,757 copying src/lighteval/tasks/tasks/custom_task_classification_grammar_task.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,759 copying src/lighteval/tasks/tasks/arithmetic.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,762 copying src/lighteval/tasks/tasks/openbookqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,764 copying src/lighteval/tasks/tasks/bigbench_hard.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,766 copying src/lighteval/tasks/tasks/wikifact.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,770 copying src/lighteval/tasks/tasks/bold.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,772 copying src/lighteval/tasks/tasks/mgsm.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,775 copying src/lighteval/tasks/tasks/anli.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,777 copying src/lighteval/tasks/tasks/gsm8k.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,779 copying src/lighteval/tasks/tasks/mmlu_pro.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,781 copying src/lighteval/tasks/tasks/dyck_language.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,783 copying src/lighteval/tasks/tasks/swag.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,785 copying src/lighteval/tasks/tasks/winogrande.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,787 copying src/lighteval/tasks/tasks/triviaqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,789 copying src/lighteval/tasks/tasks/agieval.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,791 copying src/lighteval/tasks/tasks/raft.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,793 copying src/lighteval/tasks/tasks/twitterAAE.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,795 copying src/lighteval/tasks/tasks/numeracy.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,797 copying src/lighteval/tasks/tasks/lextreme.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,799 copying src/lighteval/tasks/tasks/ethics.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,801 copying src/lighteval/tasks/tasks/qa4mre.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,803 copying src/lighteval/tasks/tasks/musr.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,805 copying src/lighteval/tasks/tasks/entity_data_imputation.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,807 copying src/lighteval/tasks/tasks/mmmu_pro.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,809 copying src/lighteval/tasks/tasks/simpleqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,811 copying src/lighteval/tasks/tasks/arc_agi_2.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,813 copying src/lighteval/tasks/tasks/quac.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,814 copying src/lighteval/tasks/tasks/med.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,816 copying src/lighteval/tasks/tasks/xwinograd.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,818 copying src/lighteval/tasks/tasks/siqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,820 copying src/lighteval/tasks/tasks/synthetic_reasoning.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,822 copying src/lighteval/tasks/tasks/legal_summarization.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,824 copying src/lighteval/tasks/tasks/narrativeqa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,826 copying src/lighteval/tasks/tasks/aimo.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,828 copying src/lighteval/tasks/tasks/drop_qa.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,830 copying src/lighteval/tasks/tasks/gsm_plus.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,832 copying src/lighteval/tasks/tasks/sciq.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,833 copying src/lighteval/tasks/tasks/asdiv.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,835 copying src/lighteval/tasks/tasks/xstory_cloze.py -> build/lib/lighteval/tasks/tasks 2025-11-13T12:00:12,838 creating build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,839 copying src/lighteval/tasks/templates/copa.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,841 copying src/lighteval/tasks/templates/__init__.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,844 copying src/lighteval/tasks/templates/hellaswag.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,846 copying src/lighteval/tasks/templates/multichoice.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,848 copying src/lighteval/tasks/templates/boolq.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,850 copying src/lighteval/tasks/templates/continuation.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,852 copying src/lighteval/tasks/templates/nli.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,855 copying src/lighteval/tasks/templates/translation.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,857 copying src/lighteval/tasks/templates/qa.py -> build/lib/lighteval/tasks/templates 2025-11-13T12:00:12,859 creating build/lib/lighteval/tasks/multilingual 2025-11-13T12:00:12,860 copying src/lighteval/tasks/multilingual/__init__.py -> build/lib/lighteval/tasks/multilingual 2025-11-13T12:00:12,862 copying src/lighteval/tasks/multilingual/adapters.py -> build/lib/lighteval/tasks/multilingual 2025-11-13T12:00:12,864 creating build/lib/lighteval/tasks/tasks/tiny_benchmarks 2025-11-13T12:00:12,865 copying src/lighteval/tasks/tasks/tiny_benchmarks/main.py -> build/lib/lighteval/tasks/tasks/tiny_benchmarks 2025-11-13T12:00:12,868 creating build/lib/lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:12,869 copying src/lighteval/tasks/tasks/mix_eval/prompts.py -> build/lib/lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:12,872 copying src/lighteval/tasks/tasks/mix_eval/judge_prompts.py -> build/lib/lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:12,875 copying src/lighteval/tasks/tasks/mix_eval/main.py -> build/lib/lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:12,878 creating build/lib/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:12,879 copying src/lighteval/tasks/tasks/ifeval/instructions_registry.py -> build/lib/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:12,881 copying src/lighteval/tasks/tasks/ifeval/instructions.py -> build/lib/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:12,884 copying src/lighteval/tasks/tasks/ifeval/instructions_utils.py -> build/lib/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:12,887 copying src/lighteval/tasks/tasks/ifeval/main.py -> build/lib/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:12,890 creating build/lib/lighteval/tasks/tasks/olympiade_bench 2025-11-13T12:00:12,891 copying src/lighteval/tasks/tasks/olympiade_bench/main.py -> build/lib/lighteval/tasks/tasks/olympiade_bench 2025-11-13T12:00:12,894 creating build/lib/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:12,894 copying src/lighteval/tasks/tasks/ifbench/instructions_registry.py -> build/lib/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:12,897 copying src/lighteval/tasks/tasks/ifbench/evaluation_lib.py -> build/lib/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:12,899 copying src/lighteval/tasks/tasks/ifbench/instructions.py -> build/lib/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:12,903 copying src/lighteval/tasks/tasks/ifbench/main.py -> build/lib/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:12,905 creating build/lib/lighteval/tasks/tasks/hle 2025-11-13T12:00:12,906 copying src/lighteval/tasks/tasks/hle/main.py -> build/lib/lighteval/tasks/tasks/hle 2025-11-13T12:00:12,909 creating build/lib/lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:12,910 copying src/lighteval/tasks/tasks/mt_bench/judge_prompt_templates.py -> build/lib/lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:12,913 copying src/lighteval/tasks/tasks/mt_bench/main.py -> build/lib/lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:12,915 creating build/lib/lighteval/tasks/tasks/lcb 2025-11-13T12:00:12,916 copying src/lighteval/tasks/tasks/lcb/codegen_metrics.py -> build/lib/lighteval/tasks/tasks/lcb 2025-11-13T12:00:12,919 copying src/lighteval/tasks/tasks/lcb/main.py -> build/lib/lighteval/tasks/tasks/lcb 2025-11-13T12:00:12,923 creating build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,924 copying src/lighteval/tasks/templates/utils/translation_literals.py -> build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,929 copying src/lighteval/tasks/templates/utils/__init__.py -> build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,932 copying src/lighteval/tasks/templates/utils/formatting_utils.py -> build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,934 copying src/lighteval/tasks/templates/utils/formulation.py -> build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,937 copying src/lighteval/tasks/templates/utils/adapter_utils.py -> build/lib/lighteval/tasks/templates/utils 2025-11-13T12:00:12,943 creating build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,944 copying src/lighteval/tasks/multilingual/tasks/worldtree_rus.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,946 copying src/lighteval/tasks/multilingual/tasks/xcopa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,949 copying src/lighteval/tasks/multilingual/tasks/ceval.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,951 copying src/lighteval/tasks/multilingual/tasks/m3exams.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,953 copying src/lighteval/tasks/multilingual/tasks/enem.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,956 copying src/lighteval/tasks/multilingual/tasks/xcodah.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,958 copying src/lighteval/tasks/multilingual/tasks/turkic.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,960 copying src/lighteval/tasks/multilingual/tasks/parus.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,963 copying src/lighteval/tasks/multilingual/tasks/meta_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,965 copying src/lighteval/tasks/multilingual/tasks/mintaka.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,968 copying src/lighteval/tasks/multilingual/tasks/thai_exams.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,970 copying src/lighteval/tasks/multilingual/tasks/xquad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,972 copying src/lighteval/tasks/multilingual/tasks/global_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,975 copying src/lighteval/tasks/multilingual/tasks/french.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,977 copying src/lighteval/tasks/multilingual/tasks/mlmm_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,980 copying src/lighteval/tasks/multilingual/tasks/turkish_arc.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,982 copying src/lighteval/tasks/multilingual/tasks/filipino.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,985 copying src/lighteval/tasks/multilingual/tasks/hindi_boolq.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,988 copying src/lighteval/tasks/multilingual/tasks/paws_x.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,990 copying src/lighteval/tasks/multilingual/tasks/cmnli.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,993 copying src/lighteval/tasks/multilingual/tasks/flores200.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,996 copying src/lighteval/tasks/multilingual/tasks/acva.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:12,998 copying src/lighteval/tasks/multilingual/tasks/cmath.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,000 copying src/lighteval/tasks/multilingual/tasks/ocnli.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,003 copying src/lighteval/tasks/multilingual/tasks/sber_squad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,008 copying src/lighteval/tasks/multilingual/tasks/xnli.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,223 copying src/lighteval/tasks/multilingual/tasks/afri_xnli.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,225 copying src/lighteval/tasks/multilingual/tasks/hellaswag_tur.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,227 copying src/lighteval/tasks/multilingual/tasks/openai_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,229 copying src/lighteval/tasks/multilingual/tasks/faquad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,231 copying src/lighteval/tasks/multilingual/tasks/german_rag.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,233 copying src/lighteval/tasks/multilingual/tasks/openbook_rus.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,235 copying src/lighteval/tasks/multilingual/tasks/arabic_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,237 copying src/lighteval/tasks/multilingual/tasks/tydiqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,239 copying src/lighteval/tasks/multilingual/tasks/cmrc2018.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,241 copying src/lighteval/tasks/multilingual/tasks/germanquad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,242 copying src/lighteval/tasks/multilingual/tasks/thaiqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,244 copying src/lighteval/tasks/multilingual/tasks/kenswquad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,246 copying src/lighteval/tasks/multilingual/tasks/copa_indic.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,248 copying src/lighteval/tasks/multilingual/tasks/chinese_squad.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,250 copying src/lighteval/tasks/multilingual/tasks/rcb.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,252 copying src/lighteval/tasks/multilingual/tasks/tquad_v2.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,254 copying src/lighteval/tasks/multilingual/tasks/french_boolq.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,255 copying src/lighteval/tasks/multilingual/tasks/xcsqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,257 copying src/lighteval/tasks/multilingual/tasks/chegeka.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,259 copying src/lighteval/tasks/multilingual/tasks/arabic_arc.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,261 copying src/lighteval/tasks/multilingual/tasks/mkqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,263 copying src/lighteval/tasks/multilingual/tasks/french_triviqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,265 copying src/lighteval/tasks/multilingual/tasks/fquad_v2.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,267 copying src/lighteval/tasks/multilingual/tasks/mgsm.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,269 copying src/lighteval/tasks/multilingual/tasks/openbook_ara.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,270 copying src/lighteval/tasks/multilingual/tasks/hellaswag_tel.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,272 copying src/lighteval/tasks/multilingual/tasks/serbian_eval.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,275 copying src/lighteval/tasks/multilingual/tasks/swahili_arc.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,277 copying src/lighteval/tasks/multilingual/tasks/cmmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,279 copying src/lighteval/tasks/multilingual/tasks/arabic.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,282 copying src/lighteval/tasks/multilingual/tasks/indicqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,284 copying src/lighteval/tasks/multilingual/tasks/mlmm_arc_challenge.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,285 copying src/lighteval/tasks/multilingual/tasks/hellaswag_hin.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,287 copying src/lighteval/tasks/multilingual/tasks/xstory.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,289 copying src/lighteval/tasks/multilingual/tasks/mlmm_hellaswag.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,291 copying src/lighteval/tasks/multilingual/tasks/xnli_indic.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,293 copying src/lighteval/tasks/multilingual/tasks/arcd.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,295 copying src/lighteval/tasks/multilingual/tasks/afri_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,296 copying src/lighteval/tasks/multilingual/tasks/xnli2.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,298 copying src/lighteval/tasks/multilingual/tasks/piqa_ar.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,300 copying src/lighteval/tasks/multilingual/tasks/belebele.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,302 copying src/lighteval/tasks/multilingual/tasks/turkish_mmlu.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,304 copying src/lighteval/tasks/multilingual/tasks/xwinograd.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,306 copying src/lighteval/tasks/multilingual/tasks/mlmm_truthfulqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,308 copying src/lighteval/tasks/multilingual/tasks/mlqa.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,310 copying src/lighteval/tasks/multilingual/tasks/soqal.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,312 copying src/lighteval/tasks/multilingual/tasks/mathlogicqa_rus.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,314 copying src/lighteval/tasks/multilingual/tasks/hindi_arc.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,316 copying src/lighteval/tasks/multilingual/tasks/squad_it.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,318 copying src/lighteval/tasks/multilingual/tasks/afri_mgsm.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,320 copying src/lighteval/tasks/multilingual/tasks/exams.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,322 copying src/lighteval/tasks/multilingual/tasks/oz.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,324 copying src/lighteval/tasks/multilingual/tasks/openbook_es.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,326 copying src/lighteval/tasks/multilingual/tasks/c3.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,327 copying src/lighteval/tasks/multilingual/tasks/hellaswag_tha.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,329 copying src/lighteval/tasks/multilingual/tasks/squad_es.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,331 copying src/lighteval/tasks/multilingual/tasks/oab_exams.py -> build/lib/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,334 creating build/lib/lighteval/tasks/multilingual/utils 2025-11-13T12:00:13,335 copying src/lighteval/tasks/multilingual/utils/__init__.py -> build/lib/lighteval/tasks/multilingual/utils 2025-11-13T12:00:13,336 copying src/lighteval/tasks/multilingual/utils/adapters_utils.py -> build/lib/lighteval/tasks/multilingual/utils 2025-11-13T12:00:13,339 copying src/lighteval/tasks/multilingual/utils/task_utils.py -> build/lib/lighteval/tasks/multilingual/utils 2025-11-13T12:00:13,341 creating build/lib/lighteval/models/nanotron 2025-11-13T12:00:13,342 copying src/lighteval/models/nanotron/nanotron_model.py -> build/lib/lighteval/models/nanotron 2025-11-13T12:00:13,346 creating build/lib/lighteval/models/sglang 2025-11-13T12:00:13,347 copying src/lighteval/models/sglang/sglang_model.py -> build/lib/lighteval/models/sglang 2025-11-13T12:00:13,351 creating build/lib/lighteval/models/vllm 2025-11-13T12:00:13,352 copying src/lighteval/models/vllm/vllm_model.py -> build/lib/lighteval/models/vllm 2025-11-13T12:00:13,356 creating build/lib/lighteval/models/endpoints 2025-11-13T12:00:13,356 copying src/lighteval/models/endpoints/litellm_model.py -> build/lib/lighteval/models/endpoints 2025-11-13T12:00:13,359 copying src/lighteval/models/endpoints/inference_providers_model.py -> build/lib/lighteval/models/endpoints 2025-11-13T12:00:13,361 copying src/lighteval/models/endpoints/tgi_model.py -> build/lib/lighteval/models/endpoints 2025-11-13T12:00:13,363 copying src/lighteval/models/endpoints/endpoint_model.py -> build/lib/lighteval/models/endpoints 2025-11-13T12:00:13,367 creating build/lib/lighteval/models/transformers 2025-11-13T12:00:13,368 copying src/lighteval/models/transformers/transformers_model.py -> build/lib/lighteval/models/transformers 2025-11-13T12:00:13,371 copying src/lighteval/models/transformers/delta_model.py -> build/lib/lighteval/models/transformers 2025-11-13T12:00:13,373 copying src/lighteval/models/transformers/vlm_transformers_model.py -> build/lib/lighteval/models/transformers 2025-11-13T12:00:13,376 copying src/lighteval/models/transformers/adapter_model.py -> build/lib/lighteval/models/transformers 2025-11-13T12:00:13,379 creating build/lib/lighteval/models/dummy 2025-11-13T12:00:13,379 copying src/lighteval/models/dummy/dummy_model.py -> build/lib/lighteval/models/dummy 2025-11-13T12:00:13,383 creating build/lib/lighteval/models/custom 2025-11-13T12:00:13,384 copying src/lighteval/models/custom/custom_model.py -> build/lib/lighteval/models/custom 2025-11-13T12:00:13,386 running egg_info 2025-11-13T12:00:13,399 writing src/lighteval.egg-info/PKG-INFO 2025-11-13T12:00:13,416 writing dependency_links to src/lighteval.egg-info/dependency_links.txt 2025-11-13T12:00:13,418 writing entry points to src/lighteval.egg-info/entry_points.txt 2025-11-13T12:00:13,428 writing requirements to src/lighteval.egg-info/requires.txt 2025-11-13T12:00:13,429 writing top-level names to src/lighteval.egg-info/top_level.txt 2025-11-13T12:00:13,472 reading manifest file 'src/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:13,488 reading manifest template 'MANIFEST.in' 2025-11-13T12:00:13,489 warning: no files found matching 'src/lighteval/tasks/tasks_table.jsonl' 2025-11-13T12:00:13,490 warning: no files found matching 'src/lighteval/metrics/*.jsonl' 2025-11-13T12:00:13,490 adding license file 'LICENSE' 2025-11-13T12:00:13,505 writing manifest file 'src/lighteval.egg-info/SOURCES.txt' 2025-11-13T12:00:13,551 installing to build/bdist.linux-armv7l/wheel 2025-11-13T12:00:13,552 running install 2025-11-13T12:00:13,574 running install_lib 2025-11-13T12:00:13,581 creating build/bdist.linux-armv7l/wheel 2025-11-13T12:00:13,583 creating build/bdist.linux-armv7l/wheel/lighteval 2025-11-13T12:00:13,584 copying build/lib/lighteval/main_accelerate.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,586 copying build/lib/lighteval/main_tasks.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,588 copying build/lib/lighteval/cli_args.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,590 copying build/lib/lighteval/main_nanotron.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,593 creating build/bdist.linux-armv7l/wheel/lighteval/metrics 2025-11-13T12:00:13,595 creating build/bdist.linux-armv7l/wheel/lighteval/metrics/imports 2025-11-13T12:00:13,596 copying build/lib/lighteval/metrics/imports/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/imports 2025-11-13T12:00:13,598 copying build/lib/lighteval/metrics/imports/data_stats_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/imports 2025-11-13T12:00:13,600 copying build/lib/lighteval/metrics/imports/bert_scorer.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/imports 2025-11-13T12:00:13,602 copying build/lib/lighteval/metrics/imports/data_stats_metric.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/imports 2025-11-13T12:00:13,605 copying build/lib/lighteval/metrics/imports/summac.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/imports 2025-11-13T12:00:13,607 copying build/lib/lighteval/metrics/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,609 copying build/lib/lighteval/metrics/metrics_corpus.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,611 copying build/lib/lighteval/metrics/metrics_sample.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,614 copying build/lib/lighteval/metrics/metrics.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,617 creating build/bdist.linux-armv7l/wheel/lighteval/metrics/harness_compatibility 2025-11-13T12:00:13,618 copying build/lib/lighteval/metrics/harness_compatibility/drop.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/harness_compatibility 2025-11-13T12:00:13,620 copying build/lib/lighteval/metrics/harness_compatibility/truthful_qa.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/harness_compatibility 2025-11-13T12:00:13,622 copying build/lib/lighteval/metrics/normalizations.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,625 copying build/lib/lighteval/metrics/dynamic_metrics.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,628 creating build/bdist.linux-armv7l/wheel/lighteval/metrics/utils 2025-11-13T12:00:13,629 copying build/lib/lighteval/metrics/utils/math_comparison.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,631 copying build/lib/lighteval/metrics/utils/metric_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,634 copying build/lib/lighteval/metrics/utils/linguistic_tokenizers.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,636 copying build/lib/lighteval/metrics/utils/judge_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,638 copying build/lib/lighteval/metrics/utils/extractive_match_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,641 copying build/lib/lighteval/metrics/utils/llm_as_judge.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,644 copying build/lib/lighteval/metrics/utils/stderr.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics/utils 2025-11-13T12:00:13,646 copying build/lib/lighteval/metrics/sample_preparator.py -> build/bdist.linux-armv7l/wheel/./lighteval/metrics 2025-11-13T12:00:13,648 copying build/lib/lighteval/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,650 copying build/lib/lighteval/data.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,652 copying build/lib/lighteval/main_inspect.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,655 copying build/lib/lighteval/__main__.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,657 copying build/lib/lighteval/main_sglang.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,659 copying build/lib/lighteval/pipeline.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,662 creating build/bdist.linux-armv7l/wheel/lighteval/logging 2025-11-13T12:00:13,663 copying build/lib/lighteval/logging/evaluation_tracker.py -> build/bdist.linux-armv7l/wheel/./lighteval/logging 2025-11-13T12:00:13,666 copying build/lib/lighteval/logging/info_loggers.py -> build/bdist.linux-armv7l/wheel/./lighteval/logging 2025-11-13T12:00:13,668 copying build/lib/lighteval/main_vllm.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:13,671 creating build/bdist.linux-armv7l/wheel/lighteval/tasks 2025-11-13T12:00:13,672 copying build/lib/lighteval/tasks/registry.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:13,675 copying build/lib/lighteval/tasks/requests.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:13,677 copying build/lib/lighteval/tasks/default_prompts.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:13,681 copying build/lib/lighteval/tasks/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:13,682 copying build/lib/lighteval/tasks/prompt_manager.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:13,686 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks 2025-11-13T12:00:13,687 copying build/lib/lighteval/tasks/tasks/real_toxicity_prompts.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,689 copying build/lib/lighteval/tasks/tasks/xcopa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,691 copying build/lib/lighteval/tasks/tasks/webqs.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,694 copying build/lib/lighteval/tasks/tasks/entitymatching.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,696 copying build/lib/lighteval/tasks/tasks/med_dialog.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,698 copying build/lib/lighteval/tasks/tasks/arc.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,699 copying build/lib/lighteval/tasks/tasks/legalsupport.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,701 copying build/lib/lighteval/tasks/tasks/imdb.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,703 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/tiny_benchmarks 2025-11-13T12:00:13,704 copying build/lib/lighteval/tasks/tasks/tiny_benchmarks/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/tiny_benchmarks 2025-11-13T12:00:13,707 copying build/lib/lighteval/tasks/tasks/mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,709 copying build/lib/lighteval/tasks/tasks/wikitext.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,711 copying build/lib/lighteval/tasks/tasks/unscramble.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,713 copying build/lib/lighteval/tasks/tasks/babi_qa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,715 copying build/lib/lighteval/tasks/tasks/logiqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,717 copying build/lib/lighteval/tasks/tasks/the_pile.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,719 copying build/lib/lighteval/tasks/tasks/truthfulqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,721 copying build/lib/lighteval/tasks/tasks/math_500.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,723 copying build/lib/lighteval/tasks/tasks/bigbench.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,726 copying build/lib/lighteval/tasks/tasks/mathqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,728 copying build/lib/lighteval/tasks/tasks/sacrebleu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,732 copying build/lib/lighteval/tasks/tasks/lexglue.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,735 copying build/lib/lighteval/tasks/tasks/civil_comments.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,738 copying build/lib/lighteval/tasks/tasks/hellaswag.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,740 copying build/lib/lighteval/tasks/tasks/prost.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,742 copying build/lib/lighteval/tasks/tasks/qasper.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,744 copying build/lib/lighteval/tasks/tasks/gpqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,747 copying build/lib/lighteval/tasks/tasks/summarization.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,750 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:13,751 copying build/lib/lighteval/tasks/tasks/mix_eval/prompts.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:13,754 copying build/lib/lighteval/tasks/tasks/mix_eval/judge_prompts.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:13,757 copying build/lib/lighteval/tasks/tasks/mix_eval/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/mix_eval 2025-11-13T12:00:13,760 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/ifeval 2025-11-13T12:00:13,762 copying build/lib/lighteval/tasks/tasks/ifeval/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifeval 2025-11-13T12:00:13,764 copying build/lib/lighteval/tasks/tasks/ifeval/instructions.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifeval 2025-11-13T12:00:13,768 copying build/lib/lighteval/tasks/tasks/ifeval/instructions_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifeval 2025-11-13T12:00:13,771 copying build/lib/lighteval/tasks/tasks/ifeval/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifeval 2025-11-13T12:00:13,774 copying build/lib/lighteval/tasks/tasks/covid_dialogue.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,777 copying build/lib/lighteval/tasks/tasks/natural_questions.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,779 copying build/lib/lighteval/tasks/tasks/jeopardy.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,781 copying build/lib/lighteval/tasks/tasks/pubmedqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,784 copying build/lib/lighteval/tasks/tasks/slr_bench.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,786 copying build/lib/lighteval/tasks/tasks/race_high.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,788 copying build/lib/lighteval/tasks/tasks/storycloze.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,790 copying build/lib/lighteval/tasks/tasks/coqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,793 copying build/lib/lighteval/tasks/tasks/mmlu_redux.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,795 copying build/lib/lighteval/tasks/tasks/aime.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,798 copying build/lib/lighteval/tasks/tasks/bbq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,800 copying build/lib/lighteval/tasks/tasks/lsat_qa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,803 copying build/lib/lighteval/tasks/tasks/squad_v2.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,805 copying build/lib/lighteval/tasks/tasks/math.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,807 copying build/lib/lighteval/tasks/tasks/headqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,810 copying build/lib/lighteval/tasks/tasks/boolq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,812 copying build/lib/lighteval/tasks/tasks/blimp.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,816 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/olympiade_bench 2025-11-13T12:00:13,817 copying build/lib/lighteval/tasks/tasks/olympiade_bench/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/olympiade_bench 2025-11-13T12:00:13,820 copying build/lib/lighteval/tasks/tasks/lambada.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,823 copying build/lib/lighteval/tasks/tasks/piqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,825 copying build/lib/lighteval/tasks/tasks/toxigen.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,827 copying build/lib/lighteval/tasks/tasks/commonsenseqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,829 copying build/lib/lighteval/tasks/tasks/glue.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,831 copying build/lib/lighteval/tasks/tasks/custom_task_classification_grammar_task.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,833 copying build/lib/lighteval/tasks/tasks/arithmetic.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,835 copying build/lib/lighteval/tasks/tasks/openbookqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,838 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/ifbench 2025-11-13T12:00:13,839 copying build/lib/lighteval/tasks/tasks/ifbench/instructions_registry.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifbench 2025-11-13T12:00:13,841 copying build/lib/lighteval/tasks/tasks/ifbench/evaluation_lib.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifbench 2025-11-13T12:00:13,843 copying build/lib/lighteval/tasks/tasks/ifbench/instructions.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifbench 2025-11-13T12:00:13,847 copying build/lib/lighteval/tasks/tasks/ifbench/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/ifbench 2025-11-13T12:00:13,849 copying build/lib/lighteval/tasks/tasks/bigbench_hard.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,851 copying build/lib/lighteval/tasks/tasks/wikifact.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,854 copying build/lib/lighteval/tasks/tasks/bold.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,856 copying build/lib/lighteval/tasks/tasks/mgsm.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,858 copying build/lib/lighteval/tasks/tasks/anli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,860 copying build/lib/lighteval/tasks/tasks/gsm8k.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,862 copying build/lib/lighteval/tasks/tasks/mmlu_pro.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,863 copying build/lib/lighteval/tasks/tasks/dyck_language.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,865 copying build/lib/lighteval/tasks/tasks/swag.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,867 copying build/lib/lighteval/tasks/tasks/winogrande.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,869 copying build/lib/lighteval/tasks/tasks/triviaqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,871 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/hle 2025-11-13T12:00:13,872 copying build/lib/lighteval/tasks/tasks/hle/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/hle 2025-11-13T12:00:13,875 copying build/lib/lighteval/tasks/tasks/agieval.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,877 copying build/lib/lighteval/tasks/tasks/raft.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,879 copying build/lib/lighteval/tasks/tasks/twitterAAE.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,881 copying build/lib/lighteval/tasks/tasks/numeracy.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,883 copying build/lib/lighteval/tasks/tasks/lextreme.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,885 copying build/lib/lighteval/tasks/tasks/ethics.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,887 copying build/lib/lighteval/tasks/tasks/qa4mre.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,889 copying build/lib/lighteval/tasks/tasks/musr.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,891 copying build/lib/lighteval/tasks/tasks/entity_data_imputation.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,893 copying build/lib/lighteval/tasks/tasks/mmmu_pro.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,895 copying build/lib/lighteval/tasks/tasks/simpleqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,896 copying build/lib/lighteval/tasks/tasks/arc_agi_2.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,898 copying build/lib/lighteval/tasks/tasks/quac.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,900 copying build/lib/lighteval/tasks/tasks/med.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,902 copying build/lib/lighteval/tasks/tasks/xwinograd.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,904 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:13,905 copying build/lib/lighteval/tasks/tasks/mt_bench/judge_prompt_templates.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:13,908 copying build/lib/lighteval/tasks/tasks/mt_bench/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/mt_bench 2025-11-13T12:00:13,910 copying build/lib/lighteval/tasks/tasks/siqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,911 copying build/lib/lighteval/tasks/tasks/synthetic_reasoning.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,913 copying build/lib/lighteval/tasks/tasks/legal_summarization.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,915 copying build/lib/lighteval/tasks/tasks/narrativeqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,917 copying build/lib/lighteval/tasks/tasks/aimo.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,919 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/tasks/lcb 2025-11-13T12:00:13,920 copying build/lib/lighteval/tasks/tasks/lcb/codegen_metrics.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/lcb 2025-11-13T12:00:13,923 copying build/lib/lighteval/tasks/tasks/lcb/main.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks/lcb 2025-11-13T12:00:13,925 copying build/lib/lighteval/tasks/tasks/drop_qa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,927 copying build/lib/lighteval/tasks/tasks/gsm_plus.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,929 copying build/lib/lighteval/tasks/tasks/sciq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,931 copying build/lib/lighteval/tasks/tasks/asdiv.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,933 copying build/lib/lighteval/tasks/tasks/xstory_cloze.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/tasks 2025-11-13T12:00:13,935 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/templates 2025-11-13T12:00:13,936 copying build/lib/lighteval/tasks/templates/copa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,939 copying build/lib/lighteval/tasks/templates/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,940 copying build/lib/lighteval/tasks/templates/hellaswag.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,943 copying build/lib/lighteval/tasks/templates/multichoice.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,945 copying build/lib/lighteval/tasks/templates/boolq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,947 copying build/lib/lighteval/tasks/templates/continuation.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,949 copying build/lib/lighteval/tasks/templates/nli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,952 copying build/lib/lighteval/tasks/templates/translation.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,954 copying build/lib/lighteval/tasks/templates/qa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates 2025-11-13T12:00:13,956 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/templates/utils 2025-11-13T12:00:13,957 copying build/lib/lighteval/tasks/templates/utils/translation_literals.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates/utils 2025-11-13T12:00:13,960 copying build/lib/lighteval/tasks/templates/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates/utils 2025-11-13T12:00:13,962 copying build/lib/lighteval/tasks/templates/utils/formatting_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates/utils 2025-11-13T12:00:13,964 copying build/lib/lighteval/tasks/templates/utils/formulation.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates/utils 2025-11-13T12:00:13,967 copying build/lib/lighteval/tasks/templates/utils/adapter_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/templates/utils 2025-11-13T12:00:13,969 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/multilingual 2025-11-13T12:00:13,970 copying build/lib/lighteval/tasks/multilingual/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual 2025-11-13T12:00:13,974 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,975 copying build/lib/lighteval/tasks/multilingual/tasks/worldtree_rus.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,977 copying build/lib/lighteval/tasks/multilingual/tasks/xcopa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,979 copying build/lib/lighteval/tasks/multilingual/tasks/ceval.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,981 copying build/lib/lighteval/tasks/multilingual/tasks/m3exams.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,983 copying build/lib/lighteval/tasks/multilingual/tasks/enem.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,985 copying build/lib/lighteval/tasks/multilingual/tasks/xcodah.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,987 copying build/lib/lighteval/tasks/multilingual/tasks/turkic.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,989 copying build/lib/lighteval/tasks/multilingual/tasks/parus.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,990 copying build/lib/lighteval/tasks/multilingual/tasks/meta_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,993 copying build/lib/lighteval/tasks/multilingual/tasks/mintaka.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,994 copying build/lib/lighteval/tasks/multilingual/tasks/thai_exams.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,996 copying build/lib/lighteval/tasks/multilingual/tasks/xquad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:13,998 copying build/lib/lighteval/tasks/multilingual/tasks/global_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,000 copying build/lib/lighteval/tasks/multilingual/tasks/french.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,002 copying build/lib/lighteval/tasks/multilingual/tasks/mlmm_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,004 copying build/lib/lighteval/tasks/multilingual/tasks/turkish_arc.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,006 copying build/lib/lighteval/tasks/multilingual/tasks/filipino.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,008 copying build/lib/lighteval/tasks/multilingual/tasks/hindi_boolq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,010 copying build/lib/lighteval/tasks/multilingual/tasks/paws_x.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,012 copying build/lib/lighteval/tasks/multilingual/tasks/cmnli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,014 copying build/lib/lighteval/tasks/multilingual/tasks/flores200.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,016 copying build/lib/lighteval/tasks/multilingual/tasks/acva.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,018 copying build/lib/lighteval/tasks/multilingual/tasks/cmath.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,020 copying build/lib/lighteval/tasks/multilingual/tasks/ocnli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,021 copying build/lib/lighteval/tasks/multilingual/tasks/sber_squad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,023 copying build/lib/lighteval/tasks/multilingual/tasks/xnli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,025 copying build/lib/lighteval/tasks/multilingual/tasks/afri_xnli.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,027 copying build/lib/lighteval/tasks/multilingual/tasks/hellaswag_tur.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,029 copying build/lib/lighteval/tasks/multilingual/tasks/openai_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,031 copying build/lib/lighteval/tasks/multilingual/tasks/faquad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,033 copying build/lib/lighteval/tasks/multilingual/tasks/german_rag.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,035 copying build/lib/lighteval/tasks/multilingual/tasks/openbook_rus.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,037 copying build/lib/lighteval/tasks/multilingual/tasks/arabic_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,039 copying build/lib/lighteval/tasks/multilingual/tasks/tydiqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,041 copying build/lib/lighteval/tasks/multilingual/tasks/cmrc2018.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,042 copying build/lib/lighteval/tasks/multilingual/tasks/germanquad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,044 copying build/lib/lighteval/tasks/multilingual/tasks/thaiqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,046 copying build/lib/lighteval/tasks/multilingual/tasks/kenswquad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,048 copying build/lib/lighteval/tasks/multilingual/tasks/copa_indic.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,050 copying build/lib/lighteval/tasks/multilingual/tasks/chinese_squad.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,051 copying build/lib/lighteval/tasks/multilingual/tasks/rcb.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,053 copying build/lib/lighteval/tasks/multilingual/tasks/tquad_v2.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,055 copying build/lib/lighteval/tasks/multilingual/tasks/french_boolq.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,057 copying build/lib/lighteval/tasks/multilingual/tasks/xcsqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,059 copying build/lib/lighteval/tasks/multilingual/tasks/chegeka.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,061 copying build/lib/lighteval/tasks/multilingual/tasks/arabic_arc.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,062 copying build/lib/lighteval/tasks/multilingual/tasks/mkqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,064 copying build/lib/lighteval/tasks/multilingual/tasks/french_triviqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,066 copying build/lib/lighteval/tasks/multilingual/tasks/fquad_v2.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,068 copying build/lib/lighteval/tasks/multilingual/tasks/mgsm.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,070 copying build/lib/lighteval/tasks/multilingual/tasks/openbook_ara.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,072 copying build/lib/lighteval/tasks/multilingual/tasks/hellaswag_tel.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,074 copying build/lib/lighteval/tasks/multilingual/tasks/serbian_eval.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,077 copying build/lib/lighteval/tasks/multilingual/tasks/swahili_arc.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,079 copying build/lib/lighteval/tasks/multilingual/tasks/cmmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,081 copying build/lib/lighteval/tasks/multilingual/tasks/arabic.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,083 copying build/lib/lighteval/tasks/multilingual/tasks/indicqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,085 copying build/lib/lighteval/tasks/multilingual/tasks/mlmm_arc_challenge.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,088 copying build/lib/lighteval/tasks/multilingual/tasks/hellaswag_hin.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,089 copying build/lib/lighteval/tasks/multilingual/tasks/xstory.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,091 copying build/lib/lighteval/tasks/multilingual/tasks/mlmm_hellaswag.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,093 copying build/lib/lighteval/tasks/multilingual/tasks/xnli_indic.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,095 copying build/lib/lighteval/tasks/multilingual/tasks/arcd.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,097 copying build/lib/lighteval/tasks/multilingual/tasks/afri_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,099 copying build/lib/lighteval/tasks/multilingual/tasks/xnli2.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,101 copying build/lib/lighteval/tasks/multilingual/tasks/piqa_ar.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,103 copying build/lib/lighteval/tasks/multilingual/tasks/belebele.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,105 copying build/lib/lighteval/tasks/multilingual/tasks/turkish_mmlu.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,107 copying build/lib/lighteval/tasks/multilingual/tasks/xwinograd.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,109 copying build/lib/lighteval/tasks/multilingual/tasks/mlmm_truthfulqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,111 copying build/lib/lighteval/tasks/multilingual/tasks/mlqa.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,113 copying build/lib/lighteval/tasks/multilingual/tasks/soqal.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,115 copying build/lib/lighteval/tasks/multilingual/tasks/mathlogicqa_rus.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,117 copying build/lib/lighteval/tasks/multilingual/tasks/hindi_arc.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,119 copying build/lib/lighteval/tasks/multilingual/tasks/squad_it.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,121 copying build/lib/lighteval/tasks/multilingual/tasks/afri_mgsm.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,123 copying build/lib/lighteval/tasks/multilingual/tasks/exams.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,125 copying build/lib/lighteval/tasks/multilingual/tasks/oz.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,127 copying build/lib/lighteval/tasks/multilingual/tasks/openbook_es.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,129 copying build/lib/lighteval/tasks/multilingual/tasks/c3.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,131 copying build/lib/lighteval/tasks/multilingual/tasks/hellaswag_tha.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,132 copying build/lib/lighteval/tasks/multilingual/tasks/squad_es.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,134 copying build/lib/lighteval/tasks/multilingual/tasks/oab_exams.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/tasks 2025-11-13T12:00:14,136 copying build/lib/lighteval/tasks/multilingual/adapters.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual 2025-11-13T12:00:14,139 creating build/bdist.linux-armv7l/wheel/lighteval/tasks/multilingual/utils 2025-11-13T12:00:14,140 copying build/lib/lighteval/tasks/multilingual/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/utils 2025-11-13T12:00:14,142 copying build/lib/lighteval/tasks/multilingual/utils/adapters_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/utils 2025-11-13T12:00:14,144 copying build/lib/lighteval/tasks/multilingual/utils/task_utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks/multilingual/utils 2025-11-13T12:00:14,146 copying build/lib/lighteval/tasks/lighteval_task.py -> build/bdist.linux-armv7l/wheel/./lighteval/tasks 2025-11-13T12:00:14,149 creating build/bdist.linux-armv7l/wheel/lighteval/models 2025-11-13T12:00:14,150 copying build/lib/lighteval/models/model_output.py -> build/bdist.linux-armv7l/wheel/./lighteval/models 2025-11-13T12:00:14,153 creating build/bdist.linux-armv7l/wheel/lighteval/models/nanotron 2025-11-13T12:00:14,154 copying build/lib/lighteval/models/nanotron/nanotron_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/nanotron 2025-11-13T12:00:14,157 creating build/bdist.linux-armv7l/wheel/lighteval/models/sglang 2025-11-13T12:00:14,158 copying build/lib/lighteval/models/sglang/sglang_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/sglang 2025-11-13T12:00:14,162 creating build/bdist.linux-armv7l/wheel/lighteval/models/vllm 2025-11-13T12:00:14,163 copying build/lib/lighteval/models/vllm/vllm_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/vllm 2025-11-13T12:00:14,166 creating build/bdist.linux-armv7l/wheel/lighteval/models/endpoints 2025-11-13T12:00:14,167 copying build/lib/lighteval/models/endpoints/litellm_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/endpoints 2025-11-13T12:00:14,170 copying build/lib/lighteval/models/endpoints/inference_providers_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/endpoints 2025-11-13T12:00:14,172 copying build/lib/lighteval/models/endpoints/tgi_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/endpoints 2025-11-13T12:00:14,174 copying build/lib/lighteval/models/endpoints/endpoint_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/endpoints 2025-11-13T12:00:14,177 copying build/lib/lighteval/models/model_loader.py -> build/bdist.linux-armv7l/wheel/./lighteval/models 2025-11-13T12:00:14,180 creating build/bdist.linux-armv7l/wheel/lighteval/models/transformers 2025-11-13T12:00:14,181 copying build/lib/lighteval/models/transformers/transformers_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/transformers 2025-11-13T12:00:14,184 copying build/lib/lighteval/models/transformers/delta_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/transformers 2025-11-13T12:00:14,187 copying build/lib/lighteval/models/transformers/vlm_transformers_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/transformers 2025-11-13T12:00:14,189 copying build/lib/lighteval/models/transformers/adapter_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/transformers 2025-11-13T12:00:14,192 copying build/lib/lighteval/models/utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/models 2025-11-13T12:00:14,194 creating build/bdist.linux-armv7l/wheel/lighteval/models/dummy 2025-11-13T12:00:14,195 copying build/lib/lighteval/models/dummy/dummy_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/dummy 2025-11-13T12:00:14,198 creating build/bdist.linux-armv7l/wheel/lighteval/models/custom 2025-11-13T12:00:14,199 copying build/lib/lighteval/models/custom/custom_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models/custom 2025-11-13T12:00:14,201 copying build/lib/lighteval/models/model_input.py -> build/bdist.linux-armv7l/wheel/./lighteval/models 2025-11-13T12:00:14,204 copying build/lib/lighteval/models/abstract_model.py -> build/bdist.linux-armv7l/wheel/./lighteval/models 2025-11-13T12:00:14,206 copying build/lib/lighteval/main_endpoint.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:14,208 copying build/lib/lighteval/main_custom.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:14,210 copying build/lib/lighteval/main_baseline.py -> build/bdist.linux-armv7l/wheel/./lighteval 2025-11-13T12:00:14,213 creating build/bdist.linux-armv7l/wheel/lighteval/utils 2025-11-13T12:00:14,214 copying build/lib/lighteval/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,216 copying build/lib/lighteval/utils/cache_management.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,218 copying build/lib/lighteval/utils/utils.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,220 copying build/lib/lighteval/utils/language.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,223 copying build/lib/lighteval/utils/timeout.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,224 copying build/lib/lighteval/utils/parallelism.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,226 copying build/lib/lighteval/utils/imports.py -> build/bdist.linux-armv7l/wheel/./lighteval/utils 2025-11-13T12:00:14,229 running install_egg_info 2025-11-13T12:00:14,234 Copying src/lighteval.egg-info to build/bdist.linux-armv7l/wheel/./lighteval-0.12.2-py3.11.egg-info 2025-11-13T12:00:14,246 running install_scripts 2025-11-13T12:00:14,258 creating build/bdist.linux-armv7l/wheel/lighteval-0.12.2.dist-info/WHEEL 2025-11-13T12:00:14,261 creating '/tmp/pip-wheel-ro5n5phe/.tmp-86lxtbzj/lighteval-0.12.2-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2025-11-13T12:00:14,264 adding 'lighteval/__init__.py' 2025-11-13T12:00:14,265 adding 'lighteval/__main__.py' 2025-11-13T12:00:14,267 adding 'lighteval/cli_args.py' 2025-11-13T12:00:14,269 adding 'lighteval/data.py' 2025-11-13T12:00:14,271 adding 'lighteval/main_accelerate.py' 2025-11-13T12:00:14,272 adding 'lighteval/main_baseline.py' 2025-11-13T12:00:14,274 adding 'lighteval/main_custom.py' 2025-11-13T12:00:14,276 adding 'lighteval/main_endpoint.py' 2025-11-13T12:00:14,279 adding 'lighteval/main_inspect.py' 2025-11-13T12:00:14,281 adding 'lighteval/main_nanotron.py' 2025-11-13T12:00:14,282 adding 'lighteval/main_sglang.py' 2025-11-13T12:00:14,284 adding 'lighteval/main_tasks.py' 2025-11-13T12:00:14,285 adding 'lighteval/main_vllm.py' 2025-11-13T12:00:14,287 adding 'lighteval/pipeline.py' 2025-11-13T12:00:14,292 adding 'lighteval/logging/evaluation_tracker.py' 2025-11-13T12:00:14,295 adding 'lighteval/logging/info_loggers.py' 2025-11-13T12:00:14,297 adding 'lighteval/metrics/__init__.py' 2025-11-13T12:00:14,299 adding 'lighteval/metrics/dynamic_metrics.py' 2025-11-13T12:00:14,302 adding 'lighteval/metrics/metrics.py' 2025-11-13T12:00:14,304 adding 'lighteval/metrics/metrics_corpus.py' 2025-11-13T12:00:14,310 adding 'lighteval/metrics/metrics_sample.py' 2025-11-13T12:00:14,313 adding 'lighteval/metrics/normalizations.py' 2025-11-13T12:00:14,314 adding 'lighteval/metrics/sample_preparator.py' 2025-11-13T12:00:14,317 adding 'lighteval/metrics/harness_compatibility/drop.py' 2025-11-13T12:00:14,318 adding 'lighteval/metrics/harness_compatibility/truthful_qa.py' 2025-11-13T12:00:14,320 adding 'lighteval/metrics/imports/__init__.py' 2025-11-13T12:00:14,322 adding 'lighteval/metrics/imports/bert_scorer.py' 2025-11-13T12:00:14,324 adding 'lighteval/metrics/imports/data_stats_metric.py' 2025-11-13T12:00:14,326 adding 'lighteval/metrics/imports/data_stats_utils.py' 2025-11-13T12:00:14,327 adding 'lighteval/metrics/imports/summac.py' 2025-11-13T12:00:14,331 adding 'lighteval/metrics/utils/extractive_match_utils.py' 2025-11-13T12:00:14,334 adding 'lighteval/metrics/utils/judge_utils.py' 2025-11-13T12:00:14,335 adding 'lighteval/metrics/utils/linguistic_tokenizers.py' 2025-11-13T12:00:14,338 adding 'lighteval/metrics/utils/llm_as_judge.py' 2025-11-13T12:00:14,341 adding 'lighteval/metrics/utils/math_comparison.py' 2025-11-13T12:00:14,342 adding 'lighteval/metrics/utils/metric_utils.py' 2025-11-13T12:00:14,344 adding 'lighteval/metrics/utils/stderr.py' 2025-11-13T12:00:14,346 adding 'lighteval/models/abstract_model.py' 2025-11-13T12:00:14,348 adding 'lighteval/models/model_input.py' 2025-11-13T12:00:14,350 adding 'lighteval/models/model_loader.py' 2025-11-13T12:00:14,351 adding 'lighteval/models/model_output.py' 2025-11-13T12:00:14,352 adding 'lighteval/models/utils.py' 2025-11-13T12:00:14,354 adding 'lighteval/models/custom/custom_model.py' 2025-11-13T12:00:14,356 adding 'lighteval/models/dummy/dummy_model.py' 2025-11-13T12:00:14,360 adding 'lighteval/models/endpoints/endpoint_model.py' 2025-11-13T12:00:14,362 adding 'lighteval/models/endpoints/inference_providers_model.py' 2025-11-13T12:00:14,364 adding 'lighteval/models/endpoints/litellm_model.py' 2025-11-13T12:00:14,366 adding 'lighteval/models/endpoints/tgi_model.py' 2025-11-13T12:00:14,373 adding 'lighteval/models/nanotron/nanotron_model.py' 2025-11-13T12:00:14,376 adding 'lighteval/models/sglang/sglang_model.py' 2025-11-13T12:00:14,378 adding 'lighteval/models/transformers/adapter_model.py' 2025-11-13T12:00:14,380 adding 'lighteval/models/transformers/delta_model.py' 2025-11-13T12:00:14,387 adding 'lighteval/models/transformers/transformers_model.py' 2025-11-13T12:00:14,389 adding 'lighteval/models/transformers/vlm_transformers_model.py' 2025-11-13T12:00:14,393 adding 'lighteval/models/vllm/vllm_model.py' 2025-11-13T12:00:14,395 adding 'lighteval/tasks/__init__.py' 2025-11-13T12:00:14,406 adding 'lighteval/tasks/default_prompts.py' 2025-11-13T12:00:14,409 adding 'lighteval/tasks/lighteval_task.py' 2025-11-13T12:00:14,412 adding 'lighteval/tasks/prompt_manager.py' 2025-11-13T12:00:14,414 adding 'lighteval/tasks/registry.py' 2025-11-13T12:00:14,416 adding 'lighteval/tasks/requests.py' 2025-11-13T12:00:14,418 adding 'lighteval/tasks/multilingual/__init__.py' 2025-11-13T12:00:14,420 adding 'lighteval/tasks/multilingual/adapters.py' 2025-11-13T12:00:14,423 adding 'lighteval/tasks/multilingual/tasks/acva.py' 2025-11-13T12:00:14,425 adding 'lighteval/tasks/multilingual/tasks/afri_mgsm.py' 2025-11-13T12:00:14,426 adding 'lighteval/tasks/multilingual/tasks/afri_mmlu.py' 2025-11-13T12:00:14,427 adding 'lighteval/tasks/multilingual/tasks/afri_xnli.py' 2025-11-13T12:00:14,431 adding 'lighteval/tasks/multilingual/tasks/arabic.py' 2025-11-13T12:00:14,432 adding 'lighteval/tasks/multilingual/tasks/arabic_arc.py' 2025-11-13T12:00:14,434 adding 'lighteval/tasks/multilingual/tasks/arabic_mmlu.py' 2025-11-13T12:00:14,435 adding 'lighteval/tasks/multilingual/tasks/arcd.py' 2025-11-13T12:00:14,436 adding 'lighteval/tasks/multilingual/tasks/belebele.py' 2025-11-13T12:00:14,438 adding 'lighteval/tasks/multilingual/tasks/c3.py' 2025-11-13T12:00:14,439 adding 'lighteval/tasks/multilingual/tasks/ceval.py' 2025-11-13T12:00:14,440 adding 'lighteval/tasks/multilingual/tasks/chegeka.py' 2025-11-13T12:00:14,441 adding 'lighteval/tasks/multilingual/tasks/chinese_squad.py' 2025-11-13T12:00:14,443 adding 'lighteval/tasks/multilingual/tasks/cmath.py' 2025-11-13T12:00:14,444 adding 'lighteval/tasks/multilingual/tasks/cmmlu.py' 2025-11-13T12:00:14,445 adding 'lighteval/tasks/multilingual/tasks/cmnli.py' 2025-11-13T12:00:14,447 adding 'lighteval/tasks/multilingual/tasks/cmrc2018.py' 2025-11-13T12:00:14,448 adding 'lighteval/tasks/multilingual/tasks/copa_indic.py' 2025-11-13T12:00:14,449 adding 'lighteval/tasks/multilingual/tasks/enem.py' 2025-11-13T12:00:14,451 adding 'lighteval/tasks/multilingual/tasks/exams.py' 2025-11-13T12:00:14,452 adding 'lighteval/tasks/multilingual/tasks/faquad.py' 2025-11-13T12:00:14,454 adding 'lighteval/tasks/multilingual/tasks/filipino.py' 2025-11-13T12:00:14,456 adding 'lighteval/tasks/multilingual/tasks/flores200.py' 2025-11-13T12:00:14,457 adding 'lighteval/tasks/multilingual/tasks/fquad_v2.py' 2025-11-13T12:00:14,459 adding 'lighteval/tasks/multilingual/tasks/french.py' 2025-11-13T12:00:14,460 adding 'lighteval/tasks/multilingual/tasks/french_boolq.py' 2025-11-13T12:00:14,461 adding 'lighteval/tasks/multilingual/tasks/french_triviqa.py' 2025-11-13T12:00:14,463 adding 'lighteval/tasks/multilingual/tasks/german_rag.py' 2025-11-13T12:00:14,464 adding 'lighteval/tasks/multilingual/tasks/germanquad.py' 2025-11-13T12:00:14,465 adding 'lighteval/tasks/multilingual/tasks/global_mmlu.py' 2025-11-13T12:00:14,467 adding 'lighteval/tasks/multilingual/tasks/hellaswag_hin.py' 2025-11-13T12:00:14,468 adding 'lighteval/tasks/multilingual/tasks/hellaswag_tel.py' 2025-11-13T12:00:14,469 adding 'lighteval/tasks/multilingual/tasks/hellaswag_tha.py' 2025-11-13T12:00:14,471 adding 'lighteval/tasks/multilingual/tasks/hellaswag_tur.py' 2025-11-13T12:00:14,472 adding 'lighteval/tasks/multilingual/tasks/hindi_arc.py' 2025-11-13T12:00:14,473 adding 'lighteval/tasks/multilingual/tasks/hindi_boolq.py' 2025-11-13T12:00:14,474 adding 'lighteval/tasks/multilingual/tasks/indicqa.py' 2025-11-13T12:00:14,475 adding 'lighteval/tasks/multilingual/tasks/kenswquad.py' 2025-11-13T12:00:14,477 adding 'lighteval/tasks/multilingual/tasks/m3exams.py' 2025-11-13T12:00:14,478 adding 'lighteval/tasks/multilingual/tasks/mathlogicqa_rus.py' 2025-11-13T12:00:14,479 adding 'lighteval/tasks/multilingual/tasks/meta_mmlu.py' 2025-11-13T12:00:14,481 adding 'lighteval/tasks/multilingual/tasks/mgsm.py' 2025-11-13T12:00:14,482 adding 'lighteval/tasks/multilingual/tasks/mintaka.py' 2025-11-13T12:00:14,483 adding 'lighteval/tasks/multilingual/tasks/mkqa.py' 2025-11-13T12:00:14,484 adding 'lighteval/tasks/multilingual/tasks/mlmm_arc_challenge.py' 2025-11-13T12:00:14,486 adding 'lighteval/tasks/multilingual/tasks/mlmm_hellaswag.py' 2025-11-13T12:00:14,487 adding 'lighteval/tasks/multilingual/tasks/mlmm_mmlu.py' 2025-11-13T12:00:14,488 adding 'lighteval/tasks/multilingual/tasks/mlmm_truthfulqa.py' 2025-11-13T12:00:14,490 adding 'lighteval/tasks/multilingual/tasks/mlqa.py' 2025-11-13T12:00:14,491 adding 'lighteval/tasks/multilingual/tasks/oab_exams.py' 2025-11-13T12:00:14,492 adding 'lighteval/tasks/multilingual/tasks/ocnli.py' 2025-11-13T12:00:14,493 adding 'lighteval/tasks/multilingual/tasks/openai_mmlu.py' 2025-11-13T12:00:14,495 adding 'lighteval/tasks/multilingual/tasks/openbook_ara.py' 2025-11-13T12:00:14,496 adding 'lighteval/tasks/multilingual/tasks/openbook_es.py' 2025-11-13T12:00:14,497 adding 'lighteval/tasks/multilingual/tasks/openbook_rus.py' 2025-11-13T12:00:14,498 adding 'lighteval/tasks/multilingual/tasks/oz.py' 2025-11-13T12:00:14,500 adding 'lighteval/tasks/multilingual/tasks/parus.py' 2025-11-13T12:00:14,501 adding 'lighteval/tasks/multilingual/tasks/paws_x.py' 2025-11-13T12:00:14,502 adding 'lighteval/tasks/multilingual/tasks/piqa_ar.py' 2025-11-13T12:00:14,504 adding 'lighteval/tasks/multilingual/tasks/rcb.py' 2025-11-13T12:00:14,505 adding 'lighteval/tasks/multilingual/tasks/sber_squad.py' 2025-11-13T12:00:14,508 adding 'lighteval/tasks/multilingual/tasks/serbian_eval.py' 2025-11-13T12:00:14,509 adding 'lighteval/tasks/multilingual/tasks/soqal.py' 2025-11-13T12:00:14,510 adding 'lighteval/tasks/multilingual/tasks/squad_es.py' 2025-11-13T12:00:14,512 adding 'lighteval/tasks/multilingual/tasks/squad_it.py' 2025-11-13T12:00:14,513 adding 'lighteval/tasks/multilingual/tasks/swahili_arc.py' 2025-11-13T12:00:14,514 adding 'lighteval/tasks/multilingual/tasks/thai_exams.py' 2025-11-13T12:00:14,516 adding 'lighteval/tasks/multilingual/tasks/thaiqa.py' 2025-11-13T12:00:14,517 adding 'lighteval/tasks/multilingual/tasks/tquad_v2.py' 2025-11-13T12:00:14,518 adding 'lighteval/tasks/multilingual/tasks/turkic.py' 2025-11-13T12:00:14,520 adding 'lighteval/tasks/multilingual/tasks/turkish_arc.py' 2025-11-13T12:00:14,521 adding 'lighteval/tasks/multilingual/tasks/turkish_mmlu.py' 2025-11-13T12:00:14,523 adding 'lighteval/tasks/multilingual/tasks/tydiqa.py' 2025-11-13T12:00:14,524 adding 'lighteval/tasks/multilingual/tasks/worldtree_rus.py' 2025-11-13T12:00:14,525 adding 'lighteval/tasks/multilingual/tasks/xcodah.py' 2025-11-13T12:00:14,526 adding 'lighteval/tasks/multilingual/tasks/xcopa.py' 2025-11-13T12:00:14,528 adding 'lighteval/tasks/multilingual/tasks/xcsqa.py' 2025-11-13T12:00:14,529 adding 'lighteval/tasks/multilingual/tasks/xnli.py' 2025-11-13T12:00:14,530 adding 'lighteval/tasks/multilingual/tasks/xnli2.py' 2025-11-13T12:00:14,532 adding 'lighteval/tasks/multilingual/tasks/xnli_indic.py' 2025-11-13T12:00:14,533 adding 'lighteval/tasks/multilingual/tasks/xquad.py' 2025-11-13T12:00:14,534 adding 'lighteval/tasks/multilingual/tasks/xstory.py' 2025-11-13T12:00:14,536 adding 'lighteval/tasks/multilingual/tasks/xwinograd.py' 2025-11-13T12:00:14,537 adding 'lighteval/tasks/multilingual/utils/__init__.py' 2025-11-13T12:00:14,539 adding 'lighteval/tasks/multilingual/utils/adapters_utils.py' 2025-11-13T12:00:14,540 adding 'lighteval/tasks/multilingual/utils/task_utils.py' 2025-11-13T12:00:14,543 adding 'lighteval/tasks/tasks/agieval.py' 2025-11-13T12:00:14,545 adding 'lighteval/tasks/tasks/aime.py' 2025-11-13T12:00:14,546 adding 'lighteval/tasks/tasks/aimo.py' 2025-11-13T12:00:14,547 adding 'lighteval/tasks/tasks/anli.py' 2025-11-13T12:00:14,549 adding 'lighteval/tasks/tasks/arc.py' 2025-11-13T12:00:14,550 adding 'lighteval/tasks/tasks/arc_agi_2.py' 2025-11-13T12:00:14,551 adding 'lighteval/tasks/tasks/arithmetic.py' 2025-11-13T12:00:14,553 adding 'lighteval/tasks/tasks/asdiv.py' 2025-11-13T12:00:14,554 adding 'lighteval/tasks/tasks/babi_qa.py' 2025-11-13T12:00:14,555 adding 'lighteval/tasks/tasks/bbq.py' 2025-11-13T12:00:14,559 adding 'lighteval/tasks/tasks/bigbench.py' 2025-11-13T12:00:14,561 adding 'lighteval/tasks/tasks/bigbench_hard.py' 2025-11-13T12:00:14,563 adding 'lighteval/tasks/tasks/blimp.py' 2025-11-13T12:00:14,565 adding 'lighteval/tasks/tasks/bold.py' 2025-11-13T12:00:14,566 adding 'lighteval/tasks/tasks/boolq.py' 2025-11-13T12:00:14,567 adding 'lighteval/tasks/tasks/civil_comments.py' 2025-11-13T12:00:14,569 adding 'lighteval/tasks/tasks/commonsenseqa.py' 2025-11-13T12:00:14,570 adding 'lighteval/tasks/tasks/coqa.py' 2025-11-13T12:00:14,571 adding 'lighteval/tasks/tasks/covid_dialogue.py' 2025-11-13T12:00:14,574 adding 'lighteval/tasks/tasks/custom_task_classification_grammar_task.py' 2025-11-13T12:00:14,575 adding 'lighteval/tasks/tasks/drop_qa.py' 2025-11-13T12:00:14,576 adding 'lighteval/tasks/tasks/dyck_language.py' 2025-11-13T12:00:14,578 adding 'lighteval/tasks/tasks/entity_data_imputation.py' 2025-11-13T12:00:14,579 adding 'lighteval/tasks/tasks/entitymatching.py' 2025-11-13T12:00:14,580 adding 'lighteval/tasks/tasks/ethics.py' 2025-11-13T12:00:14,582 adding 'lighteval/tasks/tasks/glue.py' 2025-11-13T12:00:14,583 adding 'lighteval/tasks/tasks/gpqa.py' 2025-11-13T12:00:14,584 adding 'lighteval/tasks/tasks/gsm8k.py' 2025-11-13T12:00:14,586 adding 'lighteval/tasks/tasks/gsm_plus.py' 2025-11-13T12:00:14,587 adding 'lighteval/tasks/tasks/headqa.py' 2025-11-13T12:00:14,589 adding 'lighteval/tasks/tasks/hellaswag.py' 2025-11-13T12:00:14,590 adding 'lighteval/tasks/tasks/imdb.py' 2025-11-13T12:00:14,591 adding 'lighteval/tasks/tasks/jeopardy.py' 2025-11-13T12:00:14,593 adding 'lighteval/tasks/tasks/lambada.py' 2025-11-13T12:00:14,594 adding 'lighteval/tasks/tasks/legal_summarization.py' 2025-11-13T12:00:14,595 adding 'lighteval/tasks/tasks/legalsupport.py' 2025-11-13T12:00:14,596 adding 'lighteval/tasks/tasks/lexglue.py' 2025-11-13T12:00:14,598 adding 'lighteval/tasks/tasks/lextreme.py' 2025-11-13T12:00:14,600 adding 'lighteval/tasks/tasks/logiqa.py' 2025-11-13T12:00:14,601 adding 'lighteval/tasks/tasks/lsat_qa.py' 2025-11-13T12:00:14,602 adding 'lighteval/tasks/tasks/math.py' 2025-11-13T12:00:14,603 adding 'lighteval/tasks/tasks/math_500.py' 2025-11-13T12:00:14,605 adding 'lighteval/tasks/tasks/mathqa.py' 2025-11-13T12:00:14,606 adding 'lighteval/tasks/tasks/med.py' 2025-11-13T12:00:14,607 adding 'lighteval/tasks/tasks/med_dialog.py' 2025-11-13T12:00:14,608 adding 'lighteval/tasks/tasks/mgsm.py' 2025-11-13T12:00:14,610 adding 'lighteval/tasks/tasks/mmlu.py' 2025-11-13T12:00:14,612 adding 'lighteval/tasks/tasks/mmlu_pro.py' 2025-11-13T12:00:14,613 adding 'lighteval/tasks/tasks/mmlu_redux.py' 2025-11-13T12:00:14,615 adding 'lighteval/tasks/tasks/mmmu_pro.py' 2025-11-13T12:00:14,616 adding 'lighteval/tasks/tasks/musr.py' 2025-11-13T12:00:14,617 adding 'lighteval/tasks/tasks/narrativeqa.py' 2025-11-13T12:00:14,618 adding 'lighteval/tasks/tasks/natural_questions.py' 2025-11-13T12:00:14,620 adding 'lighteval/tasks/tasks/numeracy.py' 2025-11-13T12:00:14,621 adding 'lighteval/tasks/tasks/openbookqa.py' 2025-11-13T12:00:14,622 adding 'lighteval/tasks/tasks/piqa.py' 2025-11-13T12:00:14,623 adding 'lighteval/tasks/tasks/prost.py' 2025-11-13T12:00:14,624 adding 'lighteval/tasks/tasks/pubmedqa.py' 2025-11-13T12:00:14,625 adding 'lighteval/tasks/tasks/qa4mre.py' 2025-11-13T12:00:14,627 adding 'lighteval/tasks/tasks/qasper.py' 2025-11-13T12:00:14,628 adding 'lighteval/tasks/tasks/quac.py' 2025-11-13T12:00:14,629 adding 'lighteval/tasks/tasks/race_high.py' 2025-11-13T12:00:14,630 adding 'lighteval/tasks/tasks/raft.py' 2025-11-13T12:00:14,632 adding 'lighteval/tasks/tasks/real_toxicity_prompts.py' 2025-11-13T12:00:14,635 adding 'lighteval/tasks/tasks/sacrebleu.py' 2025-11-13T12:00:14,637 adding 'lighteval/tasks/tasks/sciq.py' 2025-11-13T12:00:14,638 adding 'lighteval/tasks/tasks/simpleqa.py' 2025-11-13T12:00:14,639 adding 'lighteval/tasks/tasks/siqa.py' 2025-11-13T12:00:14,640 adding 'lighteval/tasks/tasks/slr_bench.py' 2025-11-13T12:00:14,642 adding 'lighteval/tasks/tasks/squad_v2.py' 2025-11-13T12:00:14,643 adding 'lighteval/tasks/tasks/storycloze.py' 2025-11-13T12:00:14,644 adding 'lighteval/tasks/tasks/summarization.py' 2025-11-13T12:00:14,646 adding 'lighteval/tasks/tasks/swag.py' 2025-11-13T12:00:14,647 adding 'lighteval/tasks/tasks/synthetic_reasoning.py' 2025-11-13T12:00:14,648 adding 'lighteval/tasks/tasks/the_pile.py' 2025-11-13T12:00:14,650 adding 'lighteval/tasks/tasks/toxigen.py' 2025-11-13T12:00:14,651 adding 'lighteval/tasks/tasks/triviaqa.py' 2025-11-13T12:00:14,652 adding 'lighteval/tasks/tasks/truthfulqa.py' 2025-11-13T12:00:14,653 adding 'lighteval/tasks/tasks/twitterAAE.py' 2025-11-13T12:00:14,655 adding 'lighteval/tasks/tasks/unscramble.py' 2025-11-13T12:00:14,656 adding 'lighteval/tasks/tasks/webqs.py' 2025-11-13T12:00:14,658 adding 'lighteval/tasks/tasks/wikifact.py' 2025-11-13T12:00:14,660 adding 'lighteval/tasks/tasks/wikitext.py' 2025-11-13T12:00:14,661 adding 'lighteval/tasks/tasks/winogrande.py' 2025-11-13T12:00:14,662 adding 'lighteval/tasks/tasks/xcopa.py' 2025-11-13T12:00:14,664 adding 'lighteval/tasks/tasks/xstory_cloze.py' 2025-11-13T12:00:14,665 adding 'lighteval/tasks/tasks/xwinograd.py' 2025-11-13T12:00:14,667 adding 'lighteval/tasks/tasks/hle/main.py' 2025-11-13T12:00:14,669 adding 'lighteval/tasks/tasks/ifbench/evaluation_lib.py' 2025-11-13T12:00:14,677 adding 'lighteval/tasks/tasks/ifbench/instructions.py' 2025-11-13T12:00:14,679 adding 'lighteval/tasks/tasks/ifbench/instructions_registry.py' 2025-11-13T12:00:14,681 adding 'lighteval/tasks/tasks/ifbench/main.py' 2025-11-13T12:00:14,686 adding 'lighteval/tasks/tasks/ifeval/instructions.py' 2025-11-13T12:00:14,688 adding 'lighteval/tasks/tasks/ifeval/instructions_registry.py' 2025-11-13T12:00:14,692 adding 'lighteval/tasks/tasks/ifeval/instructions_utils.py' 2025-11-13T12:00:14,694 adding 'lighteval/tasks/tasks/ifeval/main.py' 2025-11-13T12:00:14,697 adding 'lighteval/tasks/tasks/lcb/codegen_metrics.py' 2025-11-13T12:00:14,699 adding 'lighteval/tasks/tasks/lcb/main.py' 2025-11-13T12:00:14,702 adding 'lighteval/tasks/tasks/mix_eval/judge_prompts.py' 2025-11-13T12:00:14,703 adding 'lighteval/tasks/tasks/mix_eval/main.py' 2025-11-13T12:00:14,705 adding 'lighteval/tasks/tasks/mix_eval/prompts.py' 2025-11-13T12:00:14,707 adding 'lighteval/tasks/tasks/mt_bench/judge_prompt_templates.py' 2025-11-13T12:00:14,708 adding 'lighteval/tasks/tasks/mt_bench/main.py' 2025-11-13T12:00:14,711 adding 'lighteval/tasks/tasks/olympiade_bench/main.py' 2025-11-13T12:00:14,713 adding 'lighteval/tasks/tasks/tiny_benchmarks/main.py' 2025-11-13T12:00:14,715 adding 'lighteval/tasks/templates/__init__.py' 2025-11-13T12:00:14,716 adding 'lighteval/tasks/templates/boolq.py' 2025-11-13T12:00:14,718 adding 'lighteval/tasks/templates/continuation.py' 2025-11-13T12:00:14,719 adding 'lighteval/tasks/templates/copa.py' 2025-11-13T12:00:14,721 adding 'lighteval/tasks/templates/hellaswag.py' 2025-11-13T12:00:14,722 adding 'lighteval/tasks/templates/multichoice.py' 2025-11-13T12:00:14,724 adding 'lighteval/tasks/templates/nli.py' 2025-11-13T12:00:14,726 adding 'lighteval/tasks/templates/qa.py' 2025-11-13T12:00:14,727 adding 'lighteval/tasks/templates/translation.py' 2025-11-13T12:00:14,729 adding 'lighteval/tasks/templates/utils/__init__.py' 2025-11-13T12:00:14,730 adding 'lighteval/tasks/templates/utils/adapter_utils.py' 2025-11-13T12:00:14,732 adding 'lighteval/tasks/templates/utils/formatting_utils.py' 2025-11-13T12:00:14,733 adding 'lighteval/tasks/templates/utils/formulation.py' 2025-11-13T12:00:14,737 adding 'lighteval/tasks/templates/utils/translation_literals.py' 2025-11-13T12:00:14,739 adding 'lighteval/utils/__init__.py' 2025-11-13T12:00:14,742 adding 'lighteval/utils/cache_management.py' 2025-11-13T12:00:14,743 adding 'lighteval/utils/imports.py' 2025-11-13T12:00:14,745 adding 'lighteval/utils/language.py' 2025-11-13T12:00:14,747 adding 'lighteval/utils/parallelism.py' 2025-11-13T12:00:14,748 adding 'lighteval/utils/timeout.py' 2025-11-13T12:00:14,750 adding 'lighteval/utils/utils.py' 2025-11-13T12:00:14,753 adding 'lighteval-0.12.2.dist-info/licenses/LICENSE' 2025-11-13T12:00:14,755 adding 'lighteval-0.12.2.dist-info/METADATA' 2025-11-13T12:00:14,756 adding 'lighteval-0.12.2.dist-info/WHEEL' 2025-11-13T12:00:14,757 adding 'lighteval-0.12.2.dist-info/entry_points.txt' 2025-11-13T12:00:14,757 adding 'lighteval-0.12.2.dist-info/top_level.txt' 2025-11-13T12:00:14,762 adding 'lighteval-0.12.2.dist-info/RECORD' 2025-11-13T12:00:14,772 removing build/bdist.linux-armv7l/wheel 2025-11-13T12:00:14,934 Building wheel for lighteval (pyproject.toml): finished with status 'done' 2025-11-13T12:00:14,953 Created wheel for lighteval: filename=lighteval-0.12.2-py3-none-any.whl size=603313 sha256=2b3c24acb37b2e37accfc179105c0df325914fc7a3d2c732f510e935e9766789 2025-11-13T12:00:14,955 Stored in directory: /tmp/pip-ephem-wheel-cache-k2d737ne/wheels/25/08/3b/099a9ff95a4e111e31700126aa5bdb272b7cf305a8c02ca8e9 2025-11-13T12:00:14,978 Successfully built lighteval 2025-11-13T12:00:14,995 Removed build tracker: '/tmp/pip-build-tracker-d9di2q_b'